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ASSOCIATION   TESTS 

I.     SCOPE  OF  THE  WORK 

The  present  paper  forms  part  of  the  report  of  a  Committee  of 
the  American  Psychological  Association,  appointed  in  1906,  "to 
act  as  a  general  control  committee  on  the  subject  of  measure- 
ments." The  Committee  consists  of  Professor  James  R.  Angell, 
chairman,  and  Professors  Judd,  Pillsbury,  Seashore  and  Wood- 
worth.  This  Committee  was  authorized  to  organize  sub-commit- 
tees and  to  secure  the  assistance  of  other  members  of  the  As- 
sociation. A  sub-committee  on  association  tests  was  appointed, 
consisting  of  the  present  authors,  and  the  present  paper,  the 
report  of  this  sub-committee,  is  to  be  regarded  as  a  supplement 
to  the  "Report  of  the  Committee  of  the  American  Psychological 
Association  on  the  Standardizing  of  Procedure  in  Experimental 
Tests,"  published  in  1910  as  No.  53  of  the  Psychological 
Monographs. 

The  Association  entrusted  to  its  Committee  two  general  lines 
of  work :  "first,  the  determination  of  a  series  of  group  and  in- 
dividual tests,  with  reference  to  practical  application,  and  second, 
the  determination  of  standard  experiments  of  a  more  technical 
character."  The  sub-committee  on  association  tests  has  confined 
itself  to  the  first  of  these  two  lines  of  work.  Leaving  aside  the 
more  elaborate  procedure,  with  chronoscope  and  lip  key,  we  have 
fixed  our  attention  on  the  "tests"  so  frequently  employed  in  in- 
dividual and  pathological  psychology  for  determining  the  speed 
and  quality  of  association.  Tests  are  needed  which  shall  not 
require  elaborate  apparatus  nor  the  expenditure  of  much  time 
on  the  part  of  the  individual  tested.  Many  such  tests  are  in  use ; 
these  we  have  attempted  to  sift  and,  where  possible,  improve. 
The  manner  of  giving  the  tests  has  varied  from  one  experimenter 
to  another;  and  we  have  attempted  to  ascertain  the  advantages  and 
defects  of  the  different  procedures,  and  to  make  recommendations 
accordingly. 
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The  efforts  of  a  standardizing  committee  are  likely  to  be  re- 
garded with  disfavor  and  apprehension  in  many  quarters,  on  the 
ground  that  the  time  is  not  yet  ripe  for  stereotyping  either  the 
test  material  or  the  procedure.  It  may  be  felt  that  what  is  called 
for,  in  the  present  immature  condition  of  individual  psychology, 
is,  the  rather,  free  invention  and  the  appearance  of  as  many 
variants  as  possible.  Let  very  many  tests  be  tried,  each  new 
investigator  introducing  his  own  modification;  and  then,  the 
worthless  will  gradually  be  eliminated  and  the  fittest  will  survive. 
Admitting  the  general  justice  of  this  point  of  view,  we  still  be- 
lieve that  work  such  as  is  here  undertaken  may  be  of  service  in 
two  ways. 

First,  we  hope  that  the  tests  herein  recommended  may  find  ap- 
plication where  no  special  reason  exists  for  the  introduction  of  a 
new  test.  Often  appeals  for  tests  of  proved  value  are  heard  from 
those  who  desire  to  study  individual,  race,  sex,  child  or  patho- 
logical psychology — from  investigators  who  have  not  the  time  or 
inclination  to  devise  new  tests,  and  wrio,  moreover,  wish  to  be 
able  to  compare  their  results  on  one  class  of  subjects  with  results 
already  obtained  on  other  classes.  If  every  fresh  student  employs 
new  tests,  the  incomparability  of  the  results  entails  much  wasted 
effort.  Individual  and  class  psychology  is,  almost  of  necessity,  a 
cooperative  enterprise.  The  advantages  to  be  hoped  from  stand- 
ardization are  much  the  same  here  as  in  the  field  of  an- 
thropometry. 

Second,  it  can  scarcely  fail  to  be  true  in  psychology  as  in  all 
other  sciences  that  a  full  study  of  the  methods,  though  too  time- 
consuming  and  too  remote  from  final  results  to  be  attractive  at 
the  start,  is  certain  to  lead  to  more  reliable  results  in  the  long 
run.  In  the  field  of  association — aside  from  the  more  technical 
experiments  in  memory — the  methods  have  not  been  much  sub- 
jected to  the  kind  of  experimental  criticism  which  is  here  at- 
tempted. Usually  the  investigator  has  pressed  forward  to  the 
solution  of  his  problem,  devising  tests  that  seemed  suitable  to  his 
purpose,  and  then  abiding  by  them.  Our  concern  being,  on  the 
contrary,  exclusively  with  the  tests  themselves,  we  have  sought 
for  evidences  of  their  relative  value,  relying  at  first  on  the  ex- 
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perience  of  previous  investigators,  but  in  the  last  resort  on  re- 
newed experiment  with  this  end  in  view. 

The  tests  which  we  have  thus  selected  are  in  some  degree  ana- 
logous to  "tested  reagents"  in  chemistry.  They  make  no  claim, 
indeed,  to  be  "chemically  pure;"  that  is  to  say,  they  can  not  be 
guaranteed  to  give  a  true  measure  of  every  individual  tested. 
Any  mental  test  is  sure  to  be  vitiated  in  some  cases  either  by 
peculiarities  of  an  individual's  training  ,and  information,  or  by 
the  accidental  variations  to  which  mental  performance  is  subject 
from  moment  to  moment.  These  sources  of  error  exist  in  all 
measurements  of  intellectual  abilities.  In  the  face  of  such  diffi- 
culties, some  investigators  have  felt  it  necessary  to  retreat  from 
a  quantitative  attack  on  individual  psychology;  while  others, 
more  hopeful,  have  sought  to  neutralize  the  error  of  the  single 
measurement  by  statistical  methods.  In  the  study  of  class  differ- 
ences, they  have  relied  on  averages  from  large  groups;  and  in 
the  study  of  correlations,  they  have  endeavored  to  correct  for  the 
attenuation  resulting  from  chance  errors  in  the  single  measure- 
ments. But  either  reliance  on  the  averages  of  large  groups  or 
reliance  on  Spearman's  attenuation  formulae  is  a  reliance  on 
probability,  and  therefore  sure  to  be  justified  in  the  long  run,  but 
equally  sure  to  be  treacherous  somewhere  or  other.  Certainly, 
therefore,  it  is  wise  to  eliminate  from  the  tests  all  possible  sources 
of  error;  though  other  sources  of  error  still  remain,  yet  for  every 
defect  eliminated  there  is  an  increase  in  the  reliability  of  the  in- 
dividual measure,  and  so  of  the  final  result.  Now  most  of  the 
tests  hitherto  employed  involve  sources  of  error  which  can  be 
eliminated  once  they  are  detected  in  practise.  Many  of  these 
sources  of  error  are  little  details  in  the  construction  of  the  tests; 
for  example,  one  or  two  of  the  words  selected  as  stimuli  may 
have  been  ambiguous,  or  unfamiliar  to  many  subjects.  Our  work 
has  very  largely  consisted  in  attention  to  such  details ;  and  while 
we  cannot  hope  to  have  attained  perfection  of  detail,  we  are  sure 
that  we  have  taken  some  steps  in  that  direction. 

There  is  general  agreement,  in  practise,  as  to  what  shall  be 
included  under  the  heading  of  association  tests.  There  is  the 
"free  association"  test,  and  the  various  tests  of  "controlled  as- 
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sociation."  In  theory,  indeed,  it  is  difficult  to  draw  any  sharp 
line  between  association  and  memory,  or  intelligence,  or  reaction 
time.  Every  mental  test  involves  association;  but,  in  practise, 
the  association  test  is  regarded  as  limited  to  rather  simple  in- 
tellectual performances,  and  thus  is  distinguished  from  more 
complex  tests  of  intelligence.  On  the  other  hand,  the  association 
between  stimulus  and  response  which  comes  into  play  in  the 
simple  reaction,  or  in  the  discriminative  reaction,  is  simpler  and 
quicker-acting  than  that  of  the  "associative  reaction."  The  line 
is,  however,  not  sharply  defined,  and  we  have  included  in  our  list 
of  tests  one  or  two  (as  the  "number-checking  test")  which  might 
be  more  properly  classed  under  the  head  of  discrimination. 

The  distinction  between  association  and  memory  experiments 
is  even  harder  to  draw  in  theory,  though  in  practise  the  two  are 
well  enough  distinguished.  In  the  typical  memory  experiment 
new  associations  are  first  formed  and  later  examined  as  to  their 
strength;  whereas  the  association  experiment  deals  with  associa- 
tion already  formed,  and  does  not  control  the  process  of  their 
formation.  Herein  appears  an  obvious  deficiency  of  the  associa- 
tion experiment  as  compared  with  the  standard  experiments  on 
memory.  The  memory  experiment  deals  with  a  limited  system  of 
associations,  formed  specially  for  the  purpose  of  the  experiment 
and  under  controlled  conditions.  The  association  experiment  dips 
into  the  general  mass  of  the  individual's  associations,  formed  at 
various  times  and  under  varying  conditions,  with  varying  degrees 
of  frequency,  recency,  vividness,  emotional  and  intellectual  value ; 
and  all  these  conditions  vary  from  one  individual  to  another.  An 
experiment  in  the  formation  of  entirely  new  associations  gives  all 
individuals  an  equal  start;  but  a  test  dealing  with  previously 
formed  associations  can  not  hope  to  be  perfectly  fair.  It  aims, 
let  us  say,  to  give  a  measure  of  the  speed  of  the  individual's  as- 
sociative processes;  but  what  it  actually  measures  is,  to  a  large 
extent,  the  familiarity  of  the  particular  associations  called  for, 
and  the  freedom  of  these  associations  from  external  inter- 
ferences. 

In  the  face  of  these  difficulties,  the  association  test  may  still 
prove  of  value.    It  may  serve  any  one  of  at  least  three  purposes, 
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and  must  be  specifically  adapted  to  the  purpose  which  it  is  re- 
quired to  serve. 

( i )  A  measure  of  the  speed  of  formation  of  new  associations. 
Such  a  test  is  indistinguishable  from  an  experiment  in  memory 
or  practise;  but  we  have  included  one  such,  the  "substitution 
test." 

(2)  Mental  Diagnosis.  Here  the  fact  that  the  same  as- 
sociation may  have  very  different  values  in  different  individuals 
is  fully  recognized,  and  the  object  in  view  is  to  determine  the 
value  of  a  given  association  in  the  individual.  Besides  the 
emotional  value,  of  which  use  is  made  in  "psychoanalysis",  the 
interest  of  a  particular  association  may  be  the  object  of  inquiry, 
as  in  "Tatbestandsdiagnostik."  Also,  the  individual's  familiarity 
with  a  certain  sort  of  subject  matter,  or  with  a  given  form  of 
logical  relation,  may  be  the  thing  measured.  Thus  the  psycho- 
analytic viewpoint  in  association  tests  can  be  used,  not  only  for 
the  diagnosis  of  disturbing  ideas  and  complexes,  and  for  the 
detection  of  concealed  knowledge,  but  also  for  showing  the  lines 
of  thought  with  which  an  individual  is  conversant,  and  the  sort 
of  relationships  along  which  his  mind  habitually  moves.  These 
uses  of  the  association  tests  often  require  such  close  adaptation 
of  the  experimental  material  to  the  special  object  in  view  that  they 
cannot  easily  be  provided  for  by  a  standardized  series  of  tests. 

(3)  A  measure  of  mental  alertness.  The  speed  of  an  as- 
sociative reaction  depends  not  only  on  the  strength  of  the  as- 
sociative tendency  called  into  action — and  thus  on  the  previous 
training  of  that  association — but  also  on  the  "determining  tend- 
ency" or  "adjustment"  or  "set  of  mind."  In  controlled  associa- 
tion, the  speed  of  the  reaction  depends  on  the  efficiency  of  the 
control.  In  free  association,  also,  a  certain  adjustment  is  required 
in  order  that  the  stimulus  may  call  out  a  quick  response :  there 
must  be  a  receptive  attitude,  a  repression  of  any  train  of  thought 
that  would  interfere  with  the  speedy  apprehension  of  the  meaning 
of  the  stimulus ;  and  there  must  also  be  an  adjustment  to  give 
prompt  expression  to  the  first  idea  suggested  by  the  stimulus.  In 
a  test  of  either  free  or  controlled  associations,  calling  for  a  series 
of  responses  in  quick  succession  to  a  series  of  stimuli,  the  speed 
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of  the  performance  depends  on  maintaining  the  proper  adjustment 
throughout  the  series,  in  opposition  to  the  many  interfering 
tendencies  generated  by  the  successive  stimuli.  Periods  of  con- 
fusion are  apt  to  occur  in  the  course  of  such  a  series ;  and  when 
they  occur  they  impede  the  action  of  even  well-trained  associa- 
tions. One  cause  of  such  periods  of  confusion,  as  has  been 
abundantly  pointed  out  by  Jung  and  his  school,  lies  in  the  emo- 
tional value  of  certain  stimulus  words;  but  that  this  is  by  no 
means  the  only  cause  of  confusion  is  made  evident  in  the  color 
naming  and  similar  tests,  in  which  the  same  few  stimuli  are  re- 
peated many  times  in  chance  order.  The  associations  required 
are  here  thoroughly  familiar,  and  usually  operate  with  great 
promptness;  but  at  times  they  refuse  to  act  properly,  so  that,  in 
the  midst  of  a  series  of  rapid  reactions,  delayed  and  even  false 
reactions  occur  to  the  same  stimuli.  The  confusion  here  is  some- 
times due  to  wandering  of  the  attention  from  the  work  in  hand ; 
but  at  other  times  it  seems  to  be  due  to  interferences  generated  by 
the  performance  itself.  Whatever  may  be  the  cause  of  confusion 
in  each  particular  case,  efficiency  in  the  test  requires  such  a  de- 
gree of  control  as  will  eliminate  the  confusion.  Periods  of  con- 
fusion are  but  extreme  manifestations  of  inefficient  control ;  in  a 
minor  degree,  the  inefficiency  of  one  individual  in  comparison 
with  another  is  shown  by  uniform  slowness  of  response. 

In  order,  however,  to  make  the  association  tests  a  measure  of 
efficient  mental  control,  it  is  necessary  that  the  associations  de- 
manded shall  be  equally  familiar  to  the  individuals  compared.  In 
strictness,  it  is  impossible  to  make  sure  of  this;  for  the  experi- 
menter has  no  sufficient  knowledge  of  the  frequency,  recency,  etc., 
of  the  training  which  the  several  associations  have  received.  The 
best  that  can  be  done  is  to  call  only  for  such  associations  as  are 
familiar  to  all,  or  at  least  to  the  class  of  individuals  to  be 
tested. 

Regarded  as  a  measure  of  mental  alertness  or  efficiency  of  con- 
trol, the  association  test  should  be  susceptible  of  standardiza- 
tion ;  and  the  efforts  of  the  sub-committee  have  accordingly  been 
mostly  directed  to  this  end.  We  have  in  every  case  but  one — 
the  Kent-Rosanoff  experiment — sought  for  tests  in  which  the 


SCOPE  OF  THE  WORK  7 

speed  of  association  could  properly  be  taken  as  the  measure  of 
efficiency — tests  from  which  the  question  of  the  quality  of  the  re- 
sponses could  be  practically  eliminated.  To  this  end  we  have 
sought  to  determine,  usually  by  experiment,  what  associations 
are  so  generally  familiar  as  to  be  fair  material  for  a  test  of 
individual  differences  in  speed  of  association.  We  have  also 
studied  different  methods  of  administering  these  tests,  with  a 
view  to  contributing  towards  uniformity  of  procedure;  and  we 
have,  finally,  endeavored  to  furnish  average  results  obtained  by 
these  tests  with  one  class  of  subjects,  namely  young  adults  of 
fair  to  good  education. 

During  the  progress  of  our  work,  several  important  contribu- 
tions to  the  subject  have  appeared,  of  which  two  should  be  speci- 
ally mentioned,  those  of  Whipple1  and  of  Whitley.2  The  aim  and 
apparently  also  the  method  of  Professor  Whipple  in  preparing 
his  lists  of  tests  are  the  same  as  those  of  the  present  report;  but 
the  scope  of  his  work  is  much  more  inclusive,  and  the  present 
paper  therefore  represents  a  more  intensive  study  of  a  limited 
field.  The  lists  of  tests  here  offered  may  be  regarded  as  sup- 
plementing Whipple's  list  at  a  point  where  it  is  not  especially  full 
nor  especially  standardized. 

Dr.  Whitley's  work  is  concerned  very  largely,  though  not  ex- 
clusively, with  association  and  similar  tests;  and  her  purpose  is 
the  same  as  ours,  namely,  to  test  the  tests,  and  determine  by 
experiment  which  are  better  and  which  worse.  Her  methods  are 
however  different  from  ours,  in  that,  while  we  have  been  princi- 
pally concerned  with  the  details  of  each  test,  seeking  to  eliminate 
defects  and  sources  of  error,  she  has  taken  a  large  number  of 
tests,  as  they  stood,  and  compared  the  results  obtained  by  their 
use.  She  has  tried  many  similar  tests  on  the  same  subjects,  and 
has  moreover  repeated  the  same  test  a  number  of  times,  and  then 
has  evaluated  the  tests  by  the  following  criteria:  (i)  the  better 
tests  should  not  show  rapid  improvement  with  practise,  for  very 
rapid  improvement  indicates  that  some  device  for  dealing  with 
the  test,  or  some  adaptation  to  the  conditions  of  the  test,  is  of 

1  Manual  of  Mental  and  Physical  Tests,  Baltimore,  1910,  pp.  254-270,  312-343. 
'An  Empirical  Study  of  Certain  Tests  for  Individual  Differences.    Archives 
/   of  Psychology,  No.  18,  191 1. 
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prime  importance;  and  as  some  subjects  may  chance  to  hit  upon 
the  adaptation  or  device  at  once,  and  others  not,  the  first  trial  is 
likely  to  assign  an  individual  a  false  position  in  the  function  de- 
signed to  be  measured;  (2)  the  best  test  should  show  only  a  small 
variation  in  repeated  trials  (after  the  practise  effect  is  allowed 
for),  for  the  greater  the  variability,  the  less  reliable  is  the  single 
trial  or  the  average  of  a  few  trials;  (3)  the  best  of  a  number  of 
similar  tests  is  that  which  correlates  most  closely  with  the  aver- 
age of  them  all,  for  this  test  represents  the  fairest  sampling  of 
the  group  of  similar  mental  performances  which  it  is  desired  to 
measure.  In  point  of  method,  then,  Dr.  Whitley's  work  and 
ours  are  complementary ;  for  a  good  test  must  both  be  free  from 
minor  defects,  and  must  serve  to  indicate  the  efficiency  of  a  func- 
tion somewhat  broader  than  that  of  dealing  with  the  exact  ma- 
terial used  in  the  test.  In  regard  to  results,  it  is  not  easy  to 
compare  the  two  pieces  of  work,  so  much  depends  on  the  particu- 
lar tests  examined ;  but  we  find  agreement  at  several  points.  Dr. 
Whitley,  like  ourselves,  finds  the  use  of  written  responses  inad- 
missible in  a  test  for  speed  of  association;  her  results  also  tend 
to  give  the  preference  to  the  use  of  easy  and  simple  material,  such 
as  we  have  adopted;  and  some  of  the  tests  which  came  out  best  in 
her  comparison — such  as  an  "easy  opposites"  test,  a  "first  idea" 
test,  a  letter-checking  and  a  form-checking  test,  and  a  form- 
naming  test — are  very  similar  to  some  included  in  our  list. 


II.     QUESTIONS  OF  PROCEDURE 

I.       THE   FORM    OF   RESPONSE. 

Where  the  time  of  each  single  reaction  is  taken,  as  in  the 
classical  experiments  on  association  time,  the  response  has  almost 
always  been  a  spoken  word,  and  the  apparatus  has  measured 
the  time  to  the  beginning  of  the  vocal  utterance.  But  in  tests 
which  have  measured  the  time,  not  of  each  single  reaction,  but 
of  a  continuous  series  of  reactions,  several  forms  of  response 
have  been  used.  Spoken  words,  written  words,  written  letters, 
written  Arabic  numerals,  and  strokes  of  the  pencil,  checking  or 
cancelling  some  of  the  (visual)  stimuli,  have  all  been  used  in 
different  tests.  In  a  test  of  the  speed  of  any  mental  process,  it  is 
clear  that  the  motor  expression  necessary  for  experimental  pur- 
poses should  require  as  little  attention  as  possible  and  occupy  as 
little  time  as  possible.  None  of  the  above  mentioned  forms  of 
response  require  much  attention  from  an  educated  subject,  but 
speech  and  cancellation  have  some  advantage  in  this  respect  over 
writing.  In  respect  to  the  time  occupied  by  the  movement,  also, 
writing  is  at  a  disadvantage.  The  different  times  occupied  by 
these  various  sorts  of  motor  expression  can  be  judged  from  the 
following  results,  obtained  from  two  educated  subjects: 

Time  for  reading  (either  aloud  or  silently)  a  column 

of  20  disconnected  letters  or  Arabic  numerals.  .  6-7  sec. 
Time  for  reading  (either  aloud  or  silently)  a  column 

of  20  short  words,  with  a  total  of  22  syllables .  .     6-71/<2  sec. 

Time  for  copying  20  one-place  numbers 10-11       sec. 

Time  for  copying  20  disconnected  letters 12-13       sec- 

Time  for  copying  20  short  words,  containing  a  total 

of  80  letters 27-35       sec- 

Time  for  cancelling  each  of  a  list  of  20  letters  or 

words 6-8       sec. 

The  oral  response,  and  the  cancelling  movement,  have  there- 
fore a  great  advantage  even  over  the  writing  of  numerals. 
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So  slow  a  process  as  the  writing  of  words  could  never  be 
thought  of  as  a  suitable  form  of  response,  were  it  not  for  the 
fact,  that  when  a  series  of  stimuli,  such  as  a  column  of  numbers  to 
be  added  or  a  list  of  words  to  which  synonyms  are  required,  is 
presented  together,  the  perceptive,  associative  and  motor  pro- 
cesses overlap;  while  the  subject  is  writing  the  response  to  the 
first  stimulus,  he  is  already  dealing  with  the  second  stimulus.  If 
therefore  the  motor  response  is  such  as  to  occupy  little  time  in 
comparison  with  the  associative  process,  the  overlapping  brings 
it  about  that  the  time  for  the  series  of  responses  is  nearly  identical 
with  the  time  of  the  associative  processes  involved;  but  if  the 
motor  response  takes  a  much  longer  time  than  the  associative 
process,  the  time  of  the  series,  because  of  overlapping,  is  nearly 
identical  with  the  time  of  the  motor  processes.  Overlapping 
causes  a  disappearance  of  motor  time  in  the  first  case,  and  of  the 
association  time  in  the  second  case.  So  time-consuming  a  move- 
ment as  writing  can  only  be  used  as  an  idex  of  the  speed  of  asso- 
ciation when  the  associations  themselves  are  much  more  difficult 
and  slow  than  those  which  are  customary  in  mental  tests. 

With  all  this  admitted,  written  responses  might  still  find  a 
defender,  on  the  ground  that  the  writing  should  be  delayed  by 
any  halt  in  the  associative  process,  so  that,  on  the  average,  the 
longer  the  time  required  to  write  the  list  of  responses,  the  slower 
must  be  the  association.  This  is  probably  true ;  but  it  does  little 
to  weaken  the  objection  to  written  responses.  For,  first,  if  only 
one  individual  is  considered,  or  only  individuals  having  the 
same  speed  of  writing,' — and  if,  also,  the  various  words  to  be 
written  are  suitably  adjusted  as  to  length — then  the  longer 
writing  time  indicates  the  slower  association,  indeed,  but 
the  indication  is  far  from  sensitive,  and  fails  altogether  below 
a  certain  limit.  Thus,  for  example,  the  associations  involved  in 
reading  a  list  of  words,  and  those  involved  in  naming  colors,  are 
both  too  rapid  to  be  measured  by  aid  of  written  responses.  The 
results  of  one  well-trained  subject  may  be  given.  To  react  to  a 
series  of  20  patches  of  color  by  speaking  the  names  required  12 
seconds;  to  read  the  20  printed  color-names  required  but  6  sec- 
onds ;  but  to  write  the  names,  either  in  response  to  the  colors  or 
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in  response  to  a  list  of  the  names,  required  in  each  case  28  sec- 
onds. Here  written  responses  conceal  very  considerable  differ- 
ence in  speed  of  association.  Again,  in  case  of  the  "opposites" 
test,  a  subject  reacted  to  a  list  of  twenty  very  familiar  stimuli,  by 
speaking  the  opposites,  in  15  seconds;  to  a  slightly  less  familiar 
set  in  22  seconds ;  to  a  list  of  the  response  words,  by  reading  them, 
in  6.5  seconds;  but  to  copy  the  words  from  the  list  required  29 
seconds;  to  write  the  responses  to  the  easier  set  required  31  sec- 
onds, and  to  the  harder  set  30  seconds.  Thus  written  responses 
entirely  conceal  the  differences  in  speed  of  associations,  provided 
only  the  association  time  is  not  over  one  second;  and  that  even 
without  regard  to  variations  in  the  speed  of  writing.  When 
however  different  individuals  are  to  be  compared,  the  speed  of 
writing  must  be  considered;  and  as  this  speed  varies  at  least  in 
the  ratio  of  2  to  1,  even  in  educated  adults,  and  as  moreover, 
there  is  no  close  correlation  (as  we  have  found)  between  the 
speed  of  writing  and  the  speed  of  association  among  educated 
subjects,  it  is  clear,  in  conclusion,  that  conditions  can  scarcely  be 
so  favorable  as  to  justify  the  use  of  written  words  as  responses 
in  any  test  of  individual  differences  in  speed  of  association. 

The  case  is  not  quite  so  unfavorable  with  the  writing  of  single 
letters  or  one-place  numbers.  For  example,  it  is  easier  to  respond 
to  a  letter  by  giving  the  following  letter  than  by  giving  the 
preceding  letter;  and  this  difference  appears  in  either  oral  or 
written  responses.  (One  subject,  2  trials,  list  of  20  letters: 
Preceding  letter :  oral,  32  sec. ;  written,  35  sec.  Following  letter : 
oral,  20  sec;  written,  25  sec.)  The  writing  of  single  letters  or 
numerals  is  an  admissible  form  of  response  when  the  association 
time  is  over  a  second — provided  the  individuals  tested  are  ac- 
customed to  rapid  writing. 

2.       MEASUREMENT  OF  ASSOCIATION  TIME. 

As  already  remarked,  the  purpose  for  which  the  present  set  of 
tests  is  designed  excludes  the  use  of  elaborate  apparatus  and 
therefore  of  the  chronoscope  and  lip  key.     The  custom  of  many 
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students  of  association  time,  in  clinical  and  similar  work,  is  to 
employ  the  stop  watch,  starting  the  watch  together  with  the 
spoken  stimulus  word,  and  stopping  it  on  hearing  the  beginning 
of  the  response.  This  procedure  would  seem  to  include  the  ex- 
perimenter's simple  reaction  time  (probably  150-200  sunder  the 
conditions)  in  the  measured  time.  Moreover  there  is  no  guaran- 
tee that  the  watch  is  started  precisely  together  with  the  giving  of 
the  stimulus ;  a  degree  of  error  must  be  expected  here ;  and  more- 
over, the  fifth  of  a  second  of  the  stop  watch  is  scarcely  to  be  called 
a  fine  unit.  In  spite  of  these  objections,  the  use  of  the  stop  watch 
appears  to  be  justified  in  practise,  especially  since  the  variation  in 
association  time  is  so  great  that  significant  differences  can  usually 
be  established  even  with  a  rough  procedure. 

Another  procedure,  much  employed  when  the  chronoscope  can 
not  be  used,  is  to  expose  (visually)  a  whole  list  of  stimuli,  and 
to  require  the  subject  to  react  to  these  in  succession  and  without 
delay  between  the  separate  reactions.  The  time  is  then  taken,  not 
for  the  single  reactions,  but  for  the  whole  series.  As  the  time 
necessary  for  reacting  to  the  whole  series  is  usually  at  least  10 
seconds,  aftd  often  much  greater,  the  deficiencies  of  the  stop 
watch  are  not  serious  in  this  procedure.  As  indicated  in  the  pre- 
ceding section,  when  the  motor  reaction  requires  little  time  or 
attention,  the  overlapping  of  motor  and  central  processes  brings 
it  about  that  the  time  of  such  a  series  of  responses  is  essentially 
central  time.  If  therefore  the  total  time  of  the  series  of  reactions 
be  divided  by  the  number  of  responses  in  the  series,  the  quotient 
should  give  the  average  association  time. 

It  would  seem  possible,  indeed,  that  overlapping  should  accom- 
plish more  than  this,  and  make  the  average  association  time,  com- 
puted as  just  described,  considerably  less  than  that  obtained  with 
single  stimuli.  Cattell1  found  that  a  series  of  disconnected  words 
could  be  read  at  a  rate  of  200  a  per  word,  whereas  the  reading 
time  for  an  isolated  word  was  3601  a.  But  in  even  slightly  more 
difficult  reactions,  such  as  naming  presented  colors,  this  shorten- 
ing of  the  reaction  time,  when  a  series  of  stimuli  is  presented 
together,  does  not  appear;  but  the  average  time  comes  out  at 

1  Wundt's  Philos.  Studien,  1885,  2,  635. 
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from  600  to  12000".  The  same  holds  good  for  such  associations 
as  are  involved  in  the  opposites  test.  We  have  tested  five  individ- 
uals with  the  same  stimulus  words,  first  singly,  and  then,  several 
months  later,  in  lists.  Though  the  first  of  these  tests  should 
have  made  the  responses  somewhat  more  familiar,  only  one  of 
the  five  subjects  reacted  more  quickly  to  the  words  in  lists  than 
separately;  two  subjects  reacted  more  slowly  to  the  words  in  lists, 
and  two  showed  no  marked  or  consistent  difference.  On  the 
whole,  the  average  time  as  obtained  by  timing  lists  of  associative 
responses  is  no  less,  and  probably  somewhat  greater,  than  that  ob- 
tained from  separate  reactions. 

Some  explanation  is  demanded  by  the  failure  of  overlapping 
to  hasten  the  reaction  to  a  series  of  stimuli.  The  explanation  is 
probably  found  in  interferences  generated  in  the  course  of  a 
rapid  series  of  associations.  Many  associative  tendencies  are 
partially  aroused  by  each  stimulus  word,  and  when  no  interval 
elapses  between  the  successive  reactions,  the  tendencies  generated 
by  the  earlier  members  of  the  series  must  be  held  in  check  in 
order  to  give  free  play  to  the  associations  required  by  the  later 
stimuli.  Irrelevant  associations  enter  and  tend  to  impede  the 
progress  of  the  reactions.  Introspection  makes  this  view  seem 
probable,  for  often  the  subject  is  conscious  that  trains  of  thought, 
started  by  the  earlier  stimuli,  must  be  repressed  in  order  to  do 
justice  to  the  later  stimuli.  Sometimes  the  response  made  to  a 
stimulus  is  not  wholly  satisfactory  to  the  subject;  sometimes  a 
second  response  to  the  same  stimulus  is  suggested  immediately 
after  the  first  has  been  spoken ;  sometimes  an  interesting  idea  or 
disturbing  emotion  is  suggested  by  a  stimulus  or  by  the  response 
made  to  a  stimulus.  All  such  interferences  die  away  with  the 
lapse  of  a  few  seconds  between  the  stimuli ;  but  are  present 
in  full  force  when  no  interval  is  allowed.  Success  in  dealing 
rapidly  with  a  series  of  unrelated  stimuli  requires  a  higher  degree 
of  control  than  success  in  dealing  with  isolated  stimuli. 

This  interpretation  of  the  list  or  serial  test  is  borne  out  by  the 
following  experiment.1     The  subject  had  before  him  a  list  of 

1  We  are  indebted  to  Mr.  Franklin  B.  Pedrick  for  collaboration  in  this  ex- 
periment. It  is  intended  to  present  elsewhere  a  fuller  report  bearing  on 
the  question  of  fatigue  within  brief  periods  of  mental  work. 
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20  stimulus  words,  to  which  he  reacted  in  quickest  possible  suc- 
cession; but  the  experimenter,  instead  of  timing  simply  the  whole 
list,  took  the  time  for  each  reaction,  or,  at  least,  the  interval  be- 
tween each  two  successive  reactions.  This  was  accomplished  by 
bringing  side  by  side  on  a  revolving  drum  a  Jacquet  chronograph 
marking  fifths  of  seconds  and  an  electromagnetic  marker  connect- 
ed with  a  telegraph  key  on  which  rested  the  experimenter's  finger. 
The  experimenter  pressed  the  key  on  exposing  the  list  to  view,  and 
then  on  hearing  the  beginning  of  each  successive  response  of  the 
subject.  Thus  a  record  of  the  distribution  of  time  through  the 
series  of  responses  was  obtained,  having  an  accuracy  somewhat 
superior  to  that  obtained  ordinarily  with  the  stop  watch.  The 
method  can  not  be  employed  where  the  series  of  responses  is  very 
rapid  and  regular  (as  in  naming  colors),  for  then  a  rhythmic 
tendency  dominates  the  experimenter's  hand;  but  when  the  in- 
tervals between  responses  vary  irregularly  from  0.4  to  2  or  more 
seconds,  the  method  is  perfectly  feasible.  Nine  subjects  were  so 
tested,  each  reacting  to  20  lists  of  20  words.  The  instructions 
called  for  supraordinate,  subordinate  concepts,  etc.,  the  task  re- 
maining the  same  through  each  list  of  20  stimulus  words.  (The 
experiment  was  at  the  same  time  designed  to  indicate  the  com- 
parative difficulty  of  the  stimulus  words,  and  so  to  aid  in  selection 
of  the  best  lists.) 

In  combining  the  results  obtained  from  several  lists  and  from 
several  individuals,  with  the  object  of  determining  the  general 
distribution  of  time  throughout  the  list,  difficulty  arises  from  the 
inequal  difficulty  of  the  lists  and  from  the  unequal  speed  of  the 
individuals.  If  the  times  for  all  the  first  reactions  are  simply 
averaged,  and  so  for  all  the  second  reactions,  etc.,  the  general 
tendency  is  obscured  by  the  extraneous  variations  so  introduced. 
We  therefore  proceeded  as  follows :  Taking  one  individual's 
performance  in  response  to  one  list,  we  determined  the  distribu- 
tion of  time  throughout  this  one  list,  by  first  determining  the 
average  time  of  these  20  reactions,  and  the  average  deviation  of 
the  reactions,  and  then  expressing  the  time  of  each  reaction  as  + 
or  -  (according  as  the  time  of  this  reaction  was  greater  or  less 
than  the  average  time  of  the  twenty)  such  and  such  a  per  cent 
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of  the  average  deviation  of  the  reactions  in  that  list.  For 
example,  a  mark  of  -50  meant  that  the  time  of  a  reaction  was  50 
per  cent  of  the  average  deviation  less  than  the  average  time  for 
the  list.  The  same  process  was  repeated  with  each  of  the  20 
lists ;  and  the  marks  so  obtained  were  averaged  for  each  position 
within  a  list.  Thus  an  average  of  +  50  for  an  individual  in  the 
first  place  meant  that  his  first  reaction  occupied,  on  the  average, 
50  per  cent  of  his  average  deviation  more  than  his  average  time. 
The  same  process  was  repeated  for  each  individual,  and  the  in- 
dividual marks  were  averaged.  This  procedure,  then,  eliminates 
the  absolute  times,  and  also  the  absolute  variabilities,  and  gives 
an  average  picture  of  the  relative  distribution  of  time  throughout 
the  list  of  twenty.  The  net  result,  on  the  average  of  the  nine 
subjects,  is  as  follows : 

Position  in  list 1         23456789      10 

Av.  mark -j-35  — 50  — 38  — 22  — 7  — 17  — 25  — 32  — 16  — 10 

P.  E.  of  Av 15   6   4   3   4   6   6   4   5   7 

Position  in  list 11       12       13       14      15       16      17      18      19      20 

Av.  mark 4-14  -(-26    -\-2  +34  +20  +32  -j-i8    +6  -f22    — 4 

P.  E.  of  Av 6        6        4        4        8        5        8        5        6        4 

If  the  speed  of  reaction  were  uniform  through  the  list  of  twen- 
ty, the  average  mark  should  be  close  to  o  throughout ;  but  this  is 
not  the  fact.  In  spite  of  the  considerable  variations  and  the 
rather  large  P.E.,  there  can  be  no  doubt  that  reactions  2-9  tend 
to  be  quicker  than  the  average,  and  reactions  11-19  slower  than 
the  average.  There  is  a  slight  slackening  in  the  speed  of  reaction 
throughout  the  list.  The  first  and  last  reactions  are  exceptions  to 
this  rule;  for  the  first  is  slow,  and  the  last  is  more  rapid  than 
those  which  immediately  precede  it.  In  regard  to  the  first,  indi- 
vidual differences  are  here  very  great  and  characteristic ;  and  a 
fairly  strong  negative  correlation  (Pearson  r  =  -  0.69)  appears 
between  the  time  of  an  individual  for  the  whole  list  and  his  rela- 
tive time  for  the  first  word.  This  correlation  is  seen  in  the  accom- 
panying table. 
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Relative  time  for  first 
reaction 
Individual  Time  for  list  of  20         in  per  cent,  of  av.  deviation 

H    25.0  +179 

Br  28.2  -+-  41 

Pf 29.0  +103 

Pd    36.4  -)-  10 

R    374  +  64 

Wl    39-0  —  58 

Bn 39.4  —  36 

E   39.8  —21 

Wi    47.0  -|-  32 

Average    35.6  +  35 

The  individual  who  is  relatively  slowest  in  the  first  reaction 
reacts  more  rapidly  to  stimuli  in  series  than  to  single  stimuli ,  and 
in  this  respect  is  rather  exceptional.  Probably  he  manages  the 
"overlapping"  of  the  successive  acts  better  than  most  individuals. 
Accordingly  his  relative  slowness  in  the  first  reaction  may  be 
probably  explained  as  due  to  the  necessary  absence  of  overlapping 
at  the  start. 

The  fact,  however,  that  the  reaction  to  the  first  word  of  a  list 
is  on  the  average  slower  than  the  reaction  to  an  isolated  word 
shows  that  something  besides  overlapping  and  its  absence  are  in 
question.  Sometimes  a  subject  reported  that,  in  glancing  at  the 
beginning  of  a  list,  his  eye  had  caught  the  second  word  along  with 
the  first,  and  that  he  was  busied  with  the  reaction  to  the  second 
as  early  as  with  that  to  the  first.  It  even  happened,  occasionally, 
that  the  reaction  to  the  second  word  was  ready  before  that  to  the 
first.  This  form  of  interference,  incidental  as  it  is  to  overlapping, 
would  of  course  slacken  the  reaction  to  the  first  stimulus. 

Two  influences  operate  in  reacting  to  a  list  that  are  absent  in 
reacting  to  a  single  stimulus :  interference  and  overlapping.  The 
latter  tends  to  accelerate  the  reactions,  the  former  to  slacken  them, 
as  compared  with  a  reaction  to  an  isolated  simulus.  Overlapping 
can  not  exert  its  accelerating  effect  upon  the  first  reaction;  and 
interference,  also,  would  usually  not  become  operative  at  the  start, 
but  special  conditions,  such  as  seeing  the  second  word  simultan- 
eously with  the  first,  may  cause  interference  to  be  strongly  evi- 
denced at  the  very  start. 
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Aside  from  the  slowness  and  great  variability  of  the  first  re- 
action, the  most  salient  fact  resulting  from  the  above  experiment 
is  the  quickness  of  reactions  2-10  as  compared  with  reactions  n- 
19.  Why  should  the  speed  decrease  from  the  second  reaction  till 
near  the  close,  and  then  increase  again?  "Fatigue"  and  "end- 
spurt"  are  the  catch-words  that  readily  occur  to  mind;  but  neither 
of  them  is  specially  explanatory.  As  for  fatigue,  so  short  a  per- 
formance can  hardly  cause  much  fatigue  of  the  genuine,  metabol- 
ic sort.  Interference  seems  to  be  a  more  probable  conception. 
Each  succeeding  stimulus,  and  each  reaction,  tend  to  evoke  asso- 
ciations that  are  of  no  service  for  the  purpose  of  the  test.  These 
must  be  repressed ;  all  their  allurements  brushed  aside.  A  straight 
course  must  be  steered  in  spite  of  many  cross  currents.  As  these 
deflecting  tendencies  continually  accumulate  with  the  addition  of 
fresh  stimuli  and  reactions,  the  likelihood  of  disturbance  in- 
creases. 

The  increase  in  speed  at  the  very  close  can  probably  be  under- 
stood as  incidental  to  overlapping;  for,  though  overlapping  leads 
on  the  whole  to  increase  in  speed,  it  does  require,  at  every  moment, 
a  division  of  activity  between  two  or  more  reactions.  At  the 
close,  this  division  of  activity  ceases,  and  the  last  reaction  receives 
the  benefit  of  the  overlapping  without  any  of  the  incidental  draw- 
backs such  as  were  mentioned  above  in  relation  to  the  initial  re- 
action. 

The  interest  of  the  above  experiment,  in  connection  with  the 
matter  of  tests,  is  the  demonstration  that  the  list  test  brings  in 
factors — call  them  interference  and  overlapping,  or  call  them 
fatigue,  end-spurt,  etc. — which  are  not  present  in  reactions  to 
isolated  stimulus  words.  The  list  test  reaches  a  more  complicated 
mental  performance  and  calls  for  a  higher  degree  of  control. 

It  was  desired  to  see  whether  a  shorter  list  would  show  the 
same  time-curve  as  the  list  of  twenty,  and  whether  a  list  of  ten 
words  might  not  be  essentially  equivalent  to  ten  separate 
stimuli.  The  experiment  was  of  the  same  general  character  as 
above  described,  but  was  done  in  a  rougher  way.  Instead  of 
employing  a  rotating  drum,  the  experimenter  held  the  watch  to 
his  ear  and  with  his  pencil  made  wavy  lines  in  time  with  the  tick- 
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ing  of  the  watch — the  record  so  left  resembling  the  trace  of  a 
tuning  fork.  At  each  reaction  of  the  subject,  the  experimenter 
made  a  break  in  his  time  curve,  and  thus  recorded  the  time  of 
each  single  reaction.  The  accuracy  of  these  times  is  about  equal 
to  that  of  ordinary  stop  watch  readings.  Meanwhile,  by  consult- 
ing his  watch  face  at  the  end  of  the  list,  the  experimenter  had  the 
time  for  the  whole  list.  This  double  use  of  the  watch  can  be 
recommended  when  list-tests  are  used,  for  the  record  of  the  single 
reaction,  even  if  not  highly  accurate,  is  of  value  as  showing  how 
much  of  the  time  is  lost  in  a  few  slow  reactions,  and  as  making 
possible  the  calculation  of  the  median  as  well  as  the  average  time. 
Some  practise  is  of  course  necessary  before  the  experimenter  can 
successfully  use  this  device. 

The  results  of  this  experiment  were  treated  by  the  same  statis- 
tical method  as  above  described  for  the  preceding  experiment. 
Thirteen  subjects  served,  each  reacting  to  13-24  lists,  the  total 
number  of  lists  being  243.  The  average  distribution  of  time  in 
a  list  is  shown  in  the  following  table,  the  explanation  of  which 
is  the  same  as  given  on  p.  1 5  for  the  preceding  table. 

Position  in  list    1        2        3        4        5        6        7        8        9      10 

Av.  mark +179  —45  —33  —25  —11     —2  —19  —10    +3  —41 

P.  E.  of  Av 12        7        4        6        3        7        7        4        5        7 

It  is  quite  possible  that  the  very  long  relative  time  indicated  for 
the  first  reaction  is  in  part  an  artefact;  but  there  is  no  doubt 
that  the  first  reaction  is  slow,  and  that  the  last  reaction  is  quicker 
than  those  that  immediately  precede  it,  just  as  was  the  case  in 
the  list  of  twenty.  Further,  there  is  a  gradual  increase  of  reac- 
tion time  from  the  second  to  the  ninth  reaction.  This  slackening 
is  less  marked  than  in  the  list  of  twenty,  but  it  is  still  present  to  a 
degree.  It  appears  in  the  average  results  of  9  out  of  the  13  indi- 
viduals ;  and  the  4  who  do  not  show  a  slackening  show  no  progres- 
sive change  in  either  direction.  The  conclusion  is  that  the  same 
factors  are  operative  in  the  shorter  as  in  the  longer  list,  though 
not  to  as  high  a  degree. 

The  results  of  the  preceding  experiments  may  well  be  compared 
with  those  of  an  experiment  in  which  the  stimulus  words  were 
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presented  orally  and  separately.  Five  subjects  were  examined, 
each  16  times,  with  the  same  set  of  20  words  in  different  orders. 
The  time-curve  for  these  results  presents  an  entirely  different 
picture  from  that  obtained  in  the  preceding  experiments.  The 
first  reaction  is  not  slow,  but  on  the  average,  one  of  the  quickest ; 
the  last  reaction  is,  on  the  average,  one  of  the  slowest,  and  yet 
there  is  no  progressive  slackening  of  the  reactions  from  the  first 
to  the  last,  but  the  speed  remains,  on  the  whole,  very  uniform 
throughout.  The  difference  in  the  time  curve  of  the  two  modes 
of  procedure  is  probably  to  be  explained  by  reference  to  inter- 
ferences :  when  the  series  of  reactions  is  continuous,  interferences 
tend  to  accumulate  with  the  progress  of  the  series,  but  when  a 
brief  interval  of  rest  intervenes  between  the  successive  reactions 
the  interferences  tend  to  disappear.  It  may  be  concluded  that  the 
continuous  reaction  to  a  series  of  stimuli  is  a  more  complex  pro- 
cess than  the  reaction  to  a  single  stimulus,  and  requires  a  higher 
grade  of  control.  The  two  forms  of  test  are  not  therefore  equiva- 
lent, and  each  may  be  a  good  test;  but,  for  a  start,  preference 
should  be  given  to  the  simpler  form,  namely  to  the  reaction  to 
separate  stimuli.  Some  associations,  however,  such  as  the  naming 
of  colors  or  other  familiar  objects,  or  the  simplest  arithmetical 
associations,  are  too  rapid  to  be  timed,  singly,  by  the  stop  watch. 

There  is  another  advantage  in  the  timing  of  single  associative 
reactions  over  the  timing  of  a  series.  The  latter  method  gives 
indeed  the  average  association  time  for  the  stimuli  used,  but 
(unless  supplemented  by  some  such  device  as  employed  above  for 
getting  the  single  times)  it  shows  nothing  of  the  distribution  of 
the  association  times.  In  particular,  since  a  series  is  likely  to 
contain  a  few  reactions  much  longer  than  the  rest,  the  average 
time  is  apt  to  differ  considerably  from  the  median  or  the  mode, 
and  therefore  not  to  be  fully  typical.  The  very  slow  reactions 
are  usually  due  to  rather  special  causes,  and  their  great  influence 
on  the  average  is  undesirable.  The  best  procedure  would  seem  to 
be  that  of  timing  the  single  reactions,  and  using  the  median  as 
the  typical  measure. 

Where  a  list-test,  or  continuous  test,  is  employed,  our  experi- 
ence leads  us  to  favor  a  rather  short  list.     A  list  of  ten  stimulus 
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words  can  be  timed  with  sufficient  accuracy,  and  it  is  freer,  on 
the  whole,  from  interferences  of  a  disturbing  character.  In  spite 
of  this  judgment  in  favor  of  shorter  lists,  we  have  presented 
lists  of  considerable  length,  partly  to  provide  for  use  with  a  time 
limit — which  is  the  procedure  favored  by  Professor  Thorndike, 
Dr.  Whitley,  and  some  other  investigators  of  wide  experience — 
and  partly  to  provide  a  sufficient  list  of  stimuli  for  separate  reac- 
tions, when  that  is  the  method  adopted.  When  the  continuous 
method  is  used  with  an  amount  limit,  it  would  be  better  to  cut 
the  lists  in  half  and  take  two  readings.  Two  short  tests  are 
better  than  one  long  one,  because  the  average  of  the  two  is  freer 
from  the  influence  of  momentary  disturbances,  and  because  it 
allows  better  for  the  effects  of  adaptation  to  the  novel  conditions 
of  the  test. 

3.       INSTRUCTIONS  TO   THE    SUBJECT. 

The  necessity  of  uniform  instructions  has  often  been  insisted 
upon,  and  sometimes  the  instructions  have  been  reduced  to  a  set 
formula,  in  order  that  all  individuals  tested,  receiving  the  same 
instructions,  may  be  treated  alike.  A  set  formula  is,  however, 
no  guarantee  that  the  subjects  are  treated  alike,  for  some  may 
not  comprehend  the  formula.  With  a  rigid  form  of  instructions, 
the  test  becomes  partly  one  of  the  individual's  ability  to  under- 
stand the  instructions,  and  only  partly  a  test  of  the  function  exer- 
cised by  the  test  material.  It  would  be  better  to  provide  separate 
tests  for  ability  to  understand  instructions,  and  eliminate  this  fac- 
tor from  other  tests,  so  as  to  make  each,  as  far  as  possible,  a 
test  of  one  function.  Proper  comprehension  of  the  experiment 
by  the  subject  must  not  be  sacrificed  to  an  ideal  uniformity  of 
instruction.  It  matters  little  by  what  method  is  attained  the  uni- 
form result  of  the  understanding  of  the  experimental  task.  How- 
ever, there  can  be  little  question  that  the  best  method  for  this 
result  is  that  of  "learning  by  doing,"  and  that  the  subject  should 
learn  and  demonstrate  his  capacity  for  the  prescribed  reactions 
by  going  through  them.  Instruction  should  proceed  by  descrip- 
tion, illustration,  and  execution.  The  subject  should  first  be 
told  clearly  the  nature  of  the  test;  then  if  possible  he  should  see 
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the  operator  perform  some  example  of  it,  and  finally  he  should 
execute  samples  of  new  material  himself.  None  of  the  prelimi- 
nary samples  should  duplicate  the  actual  test  material ;  nor  should 
the  preliminary  trials  be  multiplied  beyond  what  is  necessary  to 
insure  the  understanding  of  the  test  and  the  first  strain  of  adapta- 
tion to  it.  As  a  rule  it  may  be  wise  to  allow  the  subject  to  make 
correct  reactions  to  two  samples  before  passing  to  the  actual  test. 

Samples  for  use  in  instructing  the  subject  should  be  prepared 
beforehand  and  lie  ready  to  the  experimenter's  hand.  For  his 
convenience,  it  has  seemed  best  to  us  to  provide  blanks  containing 
samples  of  most  of  the  tests — one  blank  containing  samples  for 
several  tests. 

The  number-checking  tests  and  the  directions  test — to  be  de- 
scribed later — require  not  oral  but  written  responses,  and  their  ex- 
ecution uses  up  the  blank  provided  for  each  subject,  and  also  the 
sample  used  by  each  subject.  Since  this  is  not  the  case  with  the  re- 
maining experiments,  the  instructional  material  of  these  two  tests 
is  best  kept  separate  from  that  for  the  remainder.  It  is  repro- 
duced1 on  the  accompanying  page  ("Instructional  Material  I"). 
For  the  two  forms  of  the  number-checking  test,  two  lines  of  ma- 
terial, organized  in  the  same  way  as  for  the  actual  test  blank,  are 
provided.  For  the  directions  test,  there  are  three  directions  sim- 
ilar to  those  on  its  actual  test  blank.  For  the  remainder  of  the 
tests,  the  same  blank  may  be  used  for  instructing  an  indefinite 
number  of  subjects  ("Instructional  Material  11").  For  each  of 
the  forms  of  addition  test,  successions  of  seven  figures  are  sup- 
plied, the  subject  after  verbal  instruction  and  illustration  reacting 
to  these  precisely  as  to  the  subsequent  test  material.  There  are 
three  sample  words  for  the  opposites  test,  and  also  three  sam- 
ples each  for  the  considerable  number  of  tests  of  partially  con- 
trolled association.  The  substitution  test  and  the  color  naming 
test  supply  their  own  instructional  material.  The  six  words 
provided  for  the  free  association  test  are  contained  neither  in  the 
Kent-Rosanoff"  experiment  nor  in  the  supplementary  thousand- 
word  list. 

1  Here,  and  in  some  other  cases  later  on,  the  exact  type,  etc.,  of  the  blank 
is  not  reproduced. 
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Instructional  Material  I. 


Number  Checking  Test. 

Form  A.     4587923601 741860592359608423 1 7821 30756494582763901 
76084395121947250836364570129865283940172376941850 


Form  B. 


215864 
8/6395 


381592 
269517 


826739 
712983 


967814 
368459 


371245 
326748 


942861 
258647 


Directions  test: 

1.  Write  any  number  larger  than  16. 

2.  Add  one  more  dot  to  the  largest  group 

3.  Put  a  cross  over  the  angle  that  opens  downwards 


ys 


/\ 


Instructional  Material  II. 

Addition  Test. 

Kraepelin  Form 

Constant  Increment  Form 

Opposites  Test 

4 

32 

better 

9 

47 

3 

21 

glad 

8 

53 

6 

39 

straight 

5 

28 

2 

65 

Logical  Relatio, 

n  Tests. 

vb-obj 

supraord 

subord 

pt-wh 

cut 

horse 

flower 

roof 

buy 

Paris 

lake 

tail 

bend 

potato 

game 

Germany 

wh-pt 

agt-act 

act-agt 

att-subst 

wheel 

train 

shines 

cold 

Europe 

frog 

howls 

cheap 

brush 

sun 

crawls 

narrow 

Mixed  Relations  Test. 

Box- 

square 

Orange — 

Woman — husband        Man — 

East- 

-west 

Day- 

Penny 

— copper              Nail— 

Asia— 

-China 

Europe — 

Grain- 

—sand 

Drop — 

Am — ' 

was 

Have — 

Free  Association  Test. 

fox 

cure 

apple 

quick 

fork 

grass 
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Even  after  thorough  and  apparently  successful  instructions,  it 
will  occasionally  happen  that  the  subject's  reaction  to  the  actual 
test  material  shows  at  once  that  he  is  on  the  wrong  track.  In 
such  cases  the  test  must  be  called  off.  But  if  the  experimenter  is 
provided  with  pairs  of  equivalent  tests* — as  can  always  be  accom- 
plished in  the  present  series,  either  from  the  provision  of  dupli- 
cate test  blanks,  or  from  the  cutting  of  a  long  blank  into  two,  as 
previously  (p.  20)  recommended — then  the  material  of  the  first 
blank  may  be  used  for  further  instructional  samples,  and  the  ac- 
tual test  carried  out  with  the  equivalent  blank. 

There  is  some  difficulty  in  bringing  all  subjects  to  an  equality 
in  their  attitude  towards  the  matter  of  speed.  Occasionally  it 
happens  that  an  individual  does  not  try  for  speed,  but  only  for 
accuracy  or  distinction  or  even  for  an  introspective  study  of  his 
performance.  Since  the  time  of  the  performance  is  the  impor- 
tant matter  in  utilizing  the  results,  it  is  unfortunate  when  a  good 
subject  fails  to  make  an  effort  for  speed.  We  have  seen  the  stand- 
ing of  an  individual  among  his  fellows  completely  changed,  in 
the  middle  of  a  series  of  tests,  by  his  being  informed  that  his  per- 
formance was  slower  than  the  average;  his  times  were  at  once 
cut  nearly  in  half,  while  his  accuracy  was  not  lessened.  We  re- 
commend that  the  instructions  include  some  such  statement  as  the 
following :  "The  main  thing  that  we  are  after  is  to  see  how  rap- 
idly your  mind  can  act.  You  need  not  be  afraid  to  put  on  speed, 
for  the  test  is  easy  and  you  are  not  likely  to  make  mistakes.  Of 
course,  you  should  keep  on  the  right  track  and  not  make  mis- 
takes, and  for  every  mistake  you  will  be  docked  a  little — about 
two  per  cent"  (or  one  per  cent  if  the  whole  of  long  blanks  is 
used).  The  docking  by  two  per  cent  is,  of  course,  purely  arbi- 
trary; and  it  may  be  desirable  with  some  classes  of  subjects  to 
make  a  larger  correction  for  errors ;  but  our  experience  with  these 
simple  tests  has  not  revealed  the  need  of  any  corrections  at  all  for 
errors.  It  is  probably  wise,  however,  to  mention  the  possibility 
of  errors  at  the  beginning,  and  to  have  an  understanding,  at 
least  with  mature  subjects,  as  to  the  degree  of  importance  attach- 
ed to  them. 


III.     CANCELLATION  TESTS— THE  NUMBER-CHECK- 
ING TEST 

Controlled  association  tests  are,  in  method,  somewhat  analo- 
gous to  choice  reaction  experiments.  Like  them,  they  may  in- 
volve a  certain  response  to  every  stimulus,  according  to  a  prear- 
ranged scheme  of  reaction  (the  B-method  of  Wundt),  or  they 
may  involve  one  single  reaction  to  a  single  kind  of  stimulus 
among  a  heterogeneous  group  of  stimuli,  the  C-method  of  Wundt. 
Here  the  subject  either  reacts  or  does  not,  i.  e.  has  the  choice  be- 
tween movement  and  rest.  The  present  test  is  the  only  represent- 
ative of  this  method  among  the  experiments  to  be  described.  The 
general  idea  is  the  presentation  to  the  subject  of  a  blank  upon 
which  are  printed  a  large  number  of  different  letters,  figures  or 
designs,  and  requiring  the  recognition  of  each  of  a  certain  symbol 
to  be  indicated  by  marking.    There  are  measured  the  speed  and 

accuracy  with  which  this  is  done.  This  experimental  conception 
is  not  a  novel  one,  having  had  its  origin  something  over  15  years 
ago,  and  having,  in  various  forms,  played  a  part  in  a  considerable 
number  of  subsequent  investigations.  In  company  with  the 
diverse  forms  of  material  that  are  available  for  its  perfor- 
mance, it  has  also  borne  a  diversity  of  names;  the  most  familiar 
single  form  is  probably  the  so-called  A-test,  first  mentioned  by 
Cattell  and  Farrand,  which  has  long  played  a  part  in  the  Columbia 
Freshman  Tests  as  a  measure  of  "rate  of  perception."  As  a 
rule,  it  seems  desirable  to  know  a  test  rather  by  an  individually 
descriptive  title  than  by  an  at  best  somewhat  vaguely  defined  men- 
tal function  with  which  it  may  be  related.  We  thus  offer  the 
present  attempt  at  the  standardization  of  this  method  under  the 
name  of  the  "Number-checking  Test;"  prepared  for  special  pur- 
poses in  two  forms. 

Form  A.  As  this  is,  so  far  as  we  know,  the  first  time  that  a 
form  made  up  exclusively  of  numerals  has  been  offered  for  this 
purpose,  it  may  not  be  amiss  to  give  some  account  of  the  consid- 
erations which  led  to  their  adoption.     The  use  of  ordinary  con- 
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textual  material  has  its  advantage  in  certain  individual  applica- 
tions of  the  test,  but  is  out  of  place  in  standard  form,  because  the 
content  of  the  text  employed  disturbs  the  attention  of  the  sub- 
ject to  the  experimental  task,  and  disturbs  it  in  different  ways  and 
in  different  degrees  for  different  subjects,  thereby  violating  the 
first  principles  upon  which  these  tests  are  constructed.  All  this 
quite  apart  from  its  absolutely  chaotic  arrangement  of  the  signi- 
ficant stimuli.  Pied  type  obviates  most  of  these  difficulties,  es- 
pecially if  methodically  arranged;  yet  even  here  it  is  difficult  to 
avoid  vocable  or  other  combinations  of  character  that  would  have 
significance  external  to  the  test.  Geometrical  forms  satisfy  the 
conditions  better,  but  here  arises  the  difficulty  of  rinding  a  proper 
number  of  distinct  geometrical  forms,  small  enough  for  the  re- 
quisite purpose,  and  recognizable  with  sufficient  readiness;  for 
every  effort  must  be  made  to  obviate  false  reactions.  These  con- 
siderations seemed  to  point  pretty  definitely  toward  the  use  of 
Arabic  numerals.  They  are  as  readily  recognizable  as  the  letters, 
and  the  chance  of  any  specially  suggestive  collocation  is  in- 
finitesimal in  comparison  with  the  letters..  The  number  of  the 
symbols,  10,  lends  itself  logically  to  the  use  of  100  of  each  symbol 
in  a  blank  of  1000  symbols,  which  is  as  long  as  there  is  any 
necessity  of  making  such  a  blank. 

The  general  character  of  the  blank  being  thus  determined, 
the  arrangement  of  the  material  took  place  as  follows:  Twenty 
lines  of  fifty  symbols  each,  properly  spaced  and  justified,  provided 
when  printed  in  the  regular  eight  point  type  a  printed  space 
with  suitable  margins  on  the  regular  blank  sheets  of  the  experi- 
ments. The  arrangement  of  each  line  is  such  that  it  contains  five 
each  of  the  symbols  1,2,3,4,5,6,7,8,9,0.  Each  successive  fifth  of 
the  line  (10  symbols)  contains  each  of  the  ten  different  symbols 
once.  This  arrangement  of  itself  obviates  the  occurrence  of 
"runs"  of  two  or  more  of  the  same  symbol,  except  at  the  begin- 
ning and  end  of  each  ten,  and  it  was  not  allowed  to  occur  here. 
The  first  ten  lines  of  the  blank  being  completed  in  this  way,  the 
second  ten  lines  were  constructed  by  reversing  this  arrangement. 
This  procedure  assured  the  approximate  equality  of  the  two  halves 
of  the  experiment  as  well  as  the  uniformity  in  the  distribution  of 
all  characters  throughout. 
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Special  researches  may  call  for  different  blanks  prepared  on 
the  same  principle  as  those  submitted.  A  great  number  of  blanks 
may  be  derived  from  the  present  one  by  methodically  replacing 
each  figure  by  some  other,  thus  everywhere  substituting  1  for  o, 
2  for  1,  etc.  The  test  has  many  desirable  features  as  a  measure 
of  fluctuations  in  continued  work.  Where  it  is  used  for  this 
purpose  a  much  longer  blank  would  probably  be  desirable,  in 
which  the  subject  should  check  a  greater  variety  of  numbers, 
say  all  the  odd  numbers.  The  time  for  every  line  could  be  noted, 
or  if  greater  precision  were  required,  the  time  could  be  reported 
electrically  by  the  graphic  method.1  The  experiment  should 
also  be  especially  convenient  in  studies  of  interference,  requiring 
a  subject  practised  for  one  number  to  check  different  numbers, 
and  the  like. 

With  the  above  arrangement,  similar  as  it  is  for  all  symbols,  it 
makes  no  difference,  so  far  as  the  arrangement  is  concerned, 
which  one  of  the  ten  symbols  the  subject  is  instructed  to  check. 
The  symbols  do  not  however  appear  to  be  equally  easy  and  their 
order  in  this  respect  should  follow  their  order  of  distinctiveness ; 
so  far  as  is  determined,  1  and  7  seem  to  be  the  easiest,  next  o 
and  4,  next  2,  3,  5,  8,  and  the  hardest  6,  9.  In  the  present  ex- 
periments the  symbol  o  has  uniformly  been  the  one  checked. 
Since  some  subjects  may  discover  essential  features  in  the  ar- 
rangement of  the  blank  while  others  do  not,  and  also  with  a  view 
to  obviating  omissions,  the  subject  should  be  uniformly  instructed 
that  there  are  in  each  line  five  of  the  symbol  they  are  to  check. 
This  reduces  omissions  to  an  insignificant  minimum,  if  it  does  not 
obviate  them  altogether;  and  this  advantage  probably  more  than 
compensates  for  the  occasional  delays  that  result  from  missing 
the  next  symbol  in  succession  and  having  to  go  back  and  search 
for  it.  In  practise  the  procedure  recommended  seems  the  more 
desirable  with  the  subjects  so  far  employed. 

After  the  subject  has  shown  proper  understanding  of  the  ex- 
periment by  correctly  executing  the  two  similarly  arranged  lines 
of  the  instructional  material,  he  begins  the  execution  of  the  test 

1  Cf.    von    Voss,    Ueber    die    Schwankungen    der    geistigen    Arbeitsleistung, 
Ps.  Arb.,  IT,  p.  300. 
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blank  according  to  the  directions  already  outlined.  A  pencil 
should  be  used  for  checking,  and  while  it  were  easy  to  appear 
hypercritical  in  this  respect,  even  in  so  minor  detail  as  the  charac- 
ter of  the  pencil,  approximate  uniformity  should  be  sought  for 
among  all  subjects  between  whom  direct  comparison  is  to  be 
made.  It  should  be  one  of  moderate  softness,  not  harder  than 
the  No.  2  grade,  otherwise  the  markings  may  require  more  than 
the  accustomed  writing  effort  to  make  them  properly  distinct. 
The  timing  of  the  complete  execution  of  the  blank  is  by  the  stop- 
watch, and  it  is  advisable  to  take  the  time  for  the  halves  also ;  un- 
less the  watch  is  provided  with  a  split-stop,  it  is  scarcely  feasible 
to  take  the  half-time  closer  than  the  nearest  second. 

Probably  no  other  of  the  present  tests  shows  such  irrelevant 
differences  in  the  manner  in  which  different  subjects  execute  it. 
The  check-marks  vary  from  light,  almost  haphazard  strokes,  to 
heavy,  labored  scorings.  It  is  impossible,  even  were  it  desirable, 
to  secure  uniformity  in  these  respects.  In  such  matters  the  sub- 
ject must  adopt  his  own  optimum  method.  While  it  is  not  clear 
that  any  special  advantage  accrues  from  this  procedure,  the  same 
must  apply  to  the  habit  of  some  subjects  to  check  alternate  lines 
backward.  More  significant  data  are  probably  obtained  by  merely 
noting  whether  the  subject  modifies  the  more  natural  behavior  to 
the  alternating  directions,  and  if  this  is  done  immediately.  A 
few  subjects,  having  checked  the  two  lines  of  the  instructions 
blank,  have  shown  a  tendency  to  check  only  the  first  two  lines  of 
the  experimental  blank,  and  subjects  should  clearly  understand 
that  the  entire  blank  is  to  be  gone  through.  Half  of  the  blank  is 
however  long  enough. 

On  rare  occasions  a  wrong  symbol  is  checked ;  this  seems  to  be 
more  frequent  in  the  later  stages  of  practise.  As  with  the  omis- 
sions, the  influence  of  such  errors  is  due  less  to  their  direct  dis- 
tortion of  the  final  time  than  to  the  fact  that  the  subject  is  quite 
apt  to  observe  them  and  be  disturbed  by  their  occurrence.  On  the 
basis  of  false  reactions,  the  test  has  not  shown,  in  the  writers' 
hands,  workable  individual  differences  in  the  "accuracy"  of  per- 
mance.  There  are  perfectly  distinct  differences  in  the  time  of 
performance,  and  there  seems  little  reason,  in  the  present  test, 
for  extending  the  scoring  beyond  this  single  factor. 
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In  this  respect,  the  range  of  normal  performance  in  the  test 
at  the  beginning  of  practise  would  seem  to  lie  between  100  and 
200  seconds,  with  an  average  of  133  for  the  whole  blank.  The 
subjects  averaging  this  figure  were  20  men  and  20  women  of  a 
similar  group.  It  is  worthy  of  note  that  the  women  averaged 
distinctly  faster  than  the  men,  123  as  against  145  seconds,  and 
also  somewhat  less  variable,  the  m.  v.'s  approximating  15  and 
22.  One  of  the  writers  has  usually  found  the  sex  difference  in 
variability  in  the  other  direction,  and,  as  is  brought  out  just  be- 
low, the  difference  between  this  and  the  succeeding  experiment 
was  somewhat  more  marked  in  the  women.  In  both  groups  the 
second  half  of  the  experiment  averages  slightly  faster  than  the 
first  half,  though  there  are  great  variations  in  this  respect. 

The  function  with  which  the  test  is  concerned  is  one  rather  sus- 
ceptible to  practise,  as  anyone  may  readily  discover  for  himself. 
In  a  succeeding  experiment  the  average  times  dropped  to  116  and 
134  for  the  women  and  men  respectively,  and  the  first  half  of 
the  experiment  averages  a  little  faster  in  the  women.  Much  of 
the  significance  of  the  experiment  depends  on  the  preservation  of 
the  individual  relationships  originally  indicated.  They  are  better 
preserved  in  the  men,  the  orders  corresponding  within  13%  for 
them  as  against  24%  for  the  women,  though  this  is  largely  due 
to  two  cases,  one  of  whom  rises  fourteen  places,  the  other  drop- 
ping ten.  The  function  is  evidently  one  whose  expression  in  the 
test  can  be  distorted  by  incidental  factors  that  are  as  yet  very 
imperfectly  understood,  and  the  advisability  of  the  greatest  possi- 
ble standardization  of  the  experimental  conditions  and  material 
is  only  further  emphasized. 

For  subjects  who  at  the  outset  evince  a  fairly  definite  quality 
of  performance,  practise  does  not  tend  to  any  great  alteration  of 
relative  position.  There  seems  to  be  less  individual  difference 
in  susceptibility  to  practise  than  in  the  range  of  performance  at 
the  beginning  of  practise. 

Form  B.  This  variant  is  termed  the  "number-group  checking 
test."  It  will  be  borne  in  mind  that  the  conditions  of  the  above 
discussed  experiment  have  been  frequently  altered  to  require  the 
checking  of  more  than  one  symbol.     The  most  familiar  form  of 
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this  is  the  so-called  a-t  test,  in  which  the  subject,  using  a  passage 
of  connected  prose,  checks  every  word  containing  both  the  letters 
a  and  t.  Other  tests  of  a  similar  nature  have  been  known  by  the 
names  of  the  symbols  checked.  This  is  not  the  same  as  merely 
requiring  the  subject  to  check  all  of  two  or  even  more  symbols, 
since  the  symbols  have  now  to  occur  in  a  definite  combination. 
The  original  blank  was  therefore  not  adapted  to  this  purpose, 
and  in  order  to  meet  in  an  analogous  way  the  possible  require- 
ments of  such  a  test,  a  special  form  was  constructed  for  it. 

This  test  contains  all  the  combinations  of  nine  digits  taken  six 
at  a  time.  The  number  of  such  combinations  =  9.8.7.6.5. 
4/1.2.3.4.5.6.  =84.  Each  single  digit  occurs  8C5  =  56  times; 
each  pair  of  digits  7C4  =  35  times ;  each  three,  6C3  =  20  times ; 
each  four,  5C2  =  10  times;  and  each  five,  4C±  =  4  times. 

In  preparing  the  test,  a  separate  card  was  used  for  each  of  the 
84  combinations,  which  were  first  written  in  the  order  of  the 
digits,  e.  g.,  134689,  125678,  245789,  etc.  Then  all  the  35  cards 
containing  both  of  the  digits  1  and  2  were  separated  out  of  the 
pack,  and  this  pair  of  digits  was  assigned  to  the  various  possible 
positions  in  the  group  of  six  with  approximately  equal  frequency. 
At  the  end  of  this  operation,  therefore,  there  were,  as  nearly  as 
possible,  equal  numbers  of  groups  arranged  in  the  following  ways 
(the  dots  indicating  positions  not  yet  filled)  : 
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Owing  to  the  preliminary  shuffling  of  the  cards,  the  relation 
of  these  positions  of  the  pair  1,2  to  the  remaining  composition 
of  the  combinations  was  haphazard. 

Next  all  of  the  cards  containing  the  pair  1,3  were  separated 


CANCELLATION  TESTS  3* 

out  of  the  pack  and  si  uffled,  and  the  assignment  of  the  pair  1,3 
to  its  several  possible  positions  with  equal  frequency  was  under- 
taken in  the  same  manner  as  before.  The  operation  of  pure 
chance  was  somewhat  limited  by  the  previous  assignment  of  the 
digits  1  and  2.  The  same  operation  was  repeated  with  each  pair 
of  digits,  14,  15,  16,  17,  18,  19,  23,  24,  25,  26,  27,  28, 
*9<  34.  35.  36,  37.  38,  39.  45.  46,  47.  48,  49.  56, 
57.  58.  59.  67.  68,  69,  78,  79,  89,  in  this  order.  As  a  matter 
of  course,  the  operation  of  chance  became  more  and  more  re- 
stricted as  this  work  progressed;  and  it  became  more  and  more 
difficult  and  finally  impossible  to  insure  that  the  several  positions 
of  a  pair  should  occur  with  equal  frequency.  All  that  could  be 
hoped  for  was  approximate  equality  of  arrangement  of  the  dif- 
ferent pairs;  and  a  review  of  the  result,  at  the  close  of  this 
operation,  showed  that  approximate  equality  had  been  attained. 
Each  pair  of  digits,  therefore,  will  be  found  not  far  from  once 
or  twice  in  each  of  its  30  possible  positions.  Since  it  is  believed 
that  the  chief  application  of  this  blank  will  be  for  checking  the 
groups  containing  a  given  pair  of  digits,  this  approximate  equality 
of  arrangement  of  all  the  possible  pairs  is  probably  what  is  most 
needed.  Some  care  was  however  taken  also  to  avoid  undue  re- 
petitions of  the  same  arrangements  of  groups  of  three  digits. 

After  the  internal  arrangement  of  each  combination  of  six 
digits  had  thus  been  determined,  the  cards  were  again  thoroughly 
shuffled  in  order  to  determine  a  chance  order  of  the  combinations. 
But  the  order  was  not  left  entirely  to  chance,  for  the  immediate 
recurrence  of  the  same  pair  of  digits  in  the  same  position  was 
avoided.  Such  immediate  recurrences  are  likely  to  be  noticed  and 
remembered  and  so  interfere  with  the  repeated  use  of  the  blank 
with  the  same  pair. 

The  result  of  all  the  operations  so  far  was  the  obtaining  of  a 
series  of  the  84  combinations  of  the  nine  digits,  taken  six  at  a 
time,  with  approximate  equality  in  position  of  each  pair  of  digits, 
and  approximately  chance  order  of  the  combinations.  Now  it  was 
desired  to  double  the  length  of  the  blank,  and  in  such  a  way  that 
the  second  half  of  the  blank  should  be  equivalent  to  the  first  half. 
Each  of  the  84  combinations  was  therefore  to  be  repeated  in  a 
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new  permutation,  and  the  equality  in  the  arrangement  of  pairs 
was  to  be  maintained.  This  could  be  accomplished,  without  need 
of  going  through  all  the  operations  involved  in  preparing  the  first 
half  of  the  blank,  by  simply  permuting  each  of  the  groups  of  the 
first  half  in  the  same  way.  Each  of  them  was,  in  fact,  permuted 
according  to  the  following  scheme:  2,  4,  6,  1,  3,  5.  The  second 
set  of  84  groups  thus  obtained  must  have  all  the  characters  im- 
pressed on  the  original  set,  only  with  change  of  the  absolute 
digits ;  and  there  can  be  no  duplication  between  the  two  sets.  By 
similar  schemes  of  permutation,  720  sets  in  all  could  be  obtained 
without  duplication  at  any  point. 

After  the  above  blank  had  been  printed,  it  was  checked  up  in 
every  way  to  see  whether  it  was  according  to  specifications,  and 
found  to  be  correct.  Since,  in  spite  of  the  statistical  equality  be- 
tween the  halves,  there  might  still  be  inequalities  in  practise,  due 
to  the  element  of  chance  entering  into  the  arrangement,  the  halves 
were  empirically  compared,  by  taking  the  time  for  each  half  both 
for  each  single  digit  and  for  each  pair.  Our  results  are  not,  in- 
deed, numerous  enough  to  establish  the  precise  equality  of  the 
halves,  but  they  give  no  reason  to  suppose  the  halves  different  in 
any  respect. 

Were  the  nine  digits  of  equal  perceptibility,  this  blank  would 
afford  a  large  number  of  equivalent  tests.  But  the  digits  are 
quite  unequal  in  perceptibility;  and  it  is  therefore  necessary  to 
establish  the  relative  difficulty  of  the  several  tests  by  trial.  We 
have  tried  the  following  tests :  ( 1 )  cancelling  the  groups  con- 
taining each  single  digit,  9  different  tests;  (2)  cancelling  the 
groups  containing  each  pair  of  digits,  36  different  tests;  (3)  a 
few  of  the  84  possible  tests  in  cancelling  groups  containing  three 
assigned  digits.  The  results,  though  not  as  extensive  as  could 
be  wished,  show  much  regularity  and  can  probably  be  taken  as 
indicating,  approximately,  the  relative  difficulty  of  the  several 
tests. 

(1)  Cancelling  of  groups  containing  a  specified  single  digit. 
In  this,  as  in  all  the  following  results,  one  half  of  the  blank  was 
used  at  a  time,  and  the  time  is  given  in  seconds. 

Seven  subjects,  previously  untrained  in  this  test,  were  tested 
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in  cancelling  each  of  the  nine  digits,  the  order  of  these  nine  tests 
being  different  with  different  subjects,  so  that  any  transferred 
practise  effect  from  one  digit  to  another  is  to  a  large  extent 
equalized  in  the  average  of  the  seven  subjects. 

Digits   1  2  3  4  5  6  7  8  9 

Av.  time 43.5    63.0    59.9    53.7    61.4    70.9    54.2    57.4    65.1 

A.  D 4.0      7.9      7-i      5-4      6.3      6.2      5.5      7.6     11. o 

P.   E 1.3      2.6      2.3      1.7      2.0      2.0      1.8      2.4      3.4 

Total  range   37~56  53~8i  46-74  42-63  48-78  62-82  39-67  43-69  50-78 

One  subject  made  eight  trials  with  each  digit,  showing  rather 
slight  improvement  after  the  second  round.  For  trials  3-8,  his 
times  average  as  follows : 

Digit:     I 

Av 31.0 

A.    D 2.2 

P.   E 0.7 

Much  the  easiest  digit  to  cancel  is  1.  It  is  easiest  with  every 
subject  tested,  and  in  every  trial.  If  the  times  for  the  other 
digits  are  expressed  as  per  cents  of  the  time  for  the  digit  1,  the 
following  are  the  relative  times : 

Digit    1         2         3         4         5         6         7         8         9 

Relative  time,  av.  of  7 
unpractised  subjects      100      145       138      123       141       163      125      132      150 

Relative  time,  one 
practised  subject. . .       100      147       130      125       138      164      116      121       143 

Relative  time,  3  sub- 
jects,  2nd   trial 100       156       131       129       130       163       121       125       141 

Av.  of  above,  with 
double  weight  al- 
lowed for  first  trial       100       148      134       125       138       163       122       128       146 

The  relative  times  in  the  three  sets  of  results  are  in  fairly 
close  agreement,  and  the  combination  in  the  last  line  of  the 
table  can  certainly  be  relied  on  within  a  few  per  cent.  It  is  cer- 
tain that  6  is  the  hardest  digit  to  find,  as  1  is  the  easiest.  The 
important  practical  question  is  whether  any  digits  are  of  nearly 
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equal  difficulty,  so  as  to  be  available  for  equivalent  tests.     Ap- 
proximate equivalence  is  assured  for  the  following  pairs : 

4  and  7 

3  and  5 

2  and  9 
Further  experience  with  the  tests  will  probably  show  the  need 
for  slight  corrections  in  treating  these  pairs  as  equivalent. 

When  only  one  pair  of  equivalent  tests  is  desired,  the  easiest  is 
probably  the  best,  especially  as  our  results  show  that  errors  and 
omissions  are  less  frequent  with  the  digits  that  give  shortest  times. 
Thus,  the  seven  unpractised  subjects  whose  times  are  reported 
above  gave  the  following  average  number  of  errors  (mostly 
omissions)  per  test: 

In  cancelling  for  the  digit  1 o        errors 

In  cancelling  for  the  digit  4,  7  or  8 Yz 

In  cancelling  for  the  digit  3  or  5 1 

In  cancelling  for  the  digit  2  or  9 iH 

In  cancelling  for  the  digit  6 2 

Since  the  time  measure  is  of  most  value  when  errors  are  ab- 
sent, the  digit  1  is  indicated  as  the  best  to  use,  except  when  there 
is  need  of  an  equivalent  pair  of  tests;  in  that  case,  4  and  7  are 
the  best  to  use. 

In  regard  to  a  correction  for  errors,  our  experience  has  not 
shown  the  need  of  one.  Our  subjects  have  not  seemed  to  save 
time  by  omissions,  but  the  time  has  been  about  the  same  either 
with  no  errors  or  with  one  or  two  or  even  three  omissions.  These 
subjects  were,  to  be  sure,  serious  and  attentive;  and  it  is  likely 
that  a  more  varied  experience  with  the  test  would  show  the 
desirability  of  correcting  for  errors.  We  judge  that  the  correc- 
tions should  be  small,  and  suggest  the  addition  of  2  per  cent,  of 
the  subject's  time  as  penalty  for  each  error  or  omission,1  when 
one  half  of  the  blank  is  used;  or  1  per  cent,  when  the  whole 
blank  is  used. 

(2)  Cancelling  of  groups  containing  a  specified  pair  of 
lAn  expeditious  method  of  detecting  errors  is  afforded  by  a  key  on  trans- 
parent paper,  to  be  laid  over  the  blank.  Whichever  digit  is  used,  the  number 
of  groups  to  be  checked  is  56  in  each  half  of  the  blank. 
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digits.  One  subject  has  made  six  trials  of  each  of  the  36  tests  of 
this  sort;  and  another  subject  has  made  one  trial  of  each.  The 
results  appear  to  have  sufficient  regularity  to  indicate  the  relative 
difficulty  of  the  several  pairs,  and  to  show  something  regarding 
the  mental  process  involved  in  this  form  of  test. 

The  time  occupied  in  checking  a  pair  of  digits  is  always  longer 
than  the  time  for  checking  either  digit  alone,  but  less  than  the 
sum  of  the  times  for  checking  the  digits  separately.  For  example, 
a  subject  takes  42  sees,  to  cancel  the  digit  4,  and  48  sees,  to 
cancel  9 ;  to  cancel  groups  containing  both  4  and  9  takes  him  64 
sees.,  which  is  71  per  cent,  of  the  sum  of  42  and  48.  The  time 
for  a  pair  is  closely  correlated  with  the  sum  of  the  times  for  the 
digits  of  the  pair,  and  is  usually  equal  to  about  70  per  cent,  of  this 
sum.    The  results  are  condensed  into  the  following  table. 

Ratio,  in  Per  Cents,  of  the  Time  for  Checking  a  Pair  of  Digits  to  the 
Sum  of  the  Times  for  Checking  the  Digits  Separately 
Subject  Average  P.  E.  A.  D.  Range 

J.  W.  T 67.3  0.4  3.1  60-75 

R.  S.  W.  first  trial 72.2  0.6  4.5  62-82 

R.  S.  W.  after  practise 72.5  0.4  3.0  66-79 

Since  this  "ratio"  is  fairly  uniform,  it  can  be  used,  in  connec- 
tion with  our  previous  table  of  times  for  checking  single  digits, 
to  indicate  the  approximate  times  for  checking  pairs.  Equivalent 
tests  can  be  selected  in  this  way;  among  the  tests  which  appear 
from  all  our  present  results  to  be  nearly  equivalent,  we  recommend 
the  following  two  pairs :  23  and  89.  These  have  the  advantage 
of  not  conflicting  with  the  digits  4  and  7  recommended  for  use 
when  single  digits  are  to  be  cancelled.  The  time  for  either  pair  is 
about  twice  that  for  the  single  digit  1,  or  about  one-and-a-half 
times  that  for  the  single  digit  4  or  7. 

It  is  possible,  from  comparison  of  the  results  of  the  two  sub- 
jects in  the  above  table,  that  there  are  genuine  individual  differ- 
ences in  the  "ratio,"  i.  e.,  in  the  speed  of  cancelling  pairs  as  com- 
pared with  the  speed  of  cancelling  single  digits.  Such  differences 
may  however,  be  merely  the  result  of  the  relative  degree  of  prac- 
tise in  the  two  sorts  of  test.    If  the  subject  has  gone  further  in 
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his  practise  with  pairs  than  in  his  practise  with  single  digits,  the 
ratio  will  evidently  be  small.  If  he  has  had  some  practise  with 
single  digits,  but  none  with  pairs,  his  first  experience  with  a  pair 
is  likely  to  give  a  high  ratio.  Thus,  subject  R.S.W.,  after  making 
two  trials  with  each  of  the  single  digits,  proceeded  to  try  in 
succession  each  of  the  36  pairs. 

The  average  "ratio,"  for  the  successive  quarters  of  this  series, 
was  as  follows : 

Av. 

First   quarter    76.5 

Second  quarter 71.9 

Third  quarter 72.6 

Fourth  quarter 67.9 

He  then  resumed  practise  with  the  single  digits,  and  after- 
wards returned  to  the  pairs,  obtaining  then  the  average  ratio  of 
72.5,  as  shown  above.  If  practise  is  continued  pari  passu  with 
single  digits  and  with  pairs,  the  ratio  would  probably  remain  in 
the  neighborhood  of  70  per  cent.  But  at  the  very  start,  the  time 
for  a  pair  is  likely  to  be  about  yy  per  cent,  of  the  sum  of  the 
times  for  the  single  digits ;  this  is  indicated  also  by  less  complete 
results  from  several  other  subjects. 

The  dependence  of  the  "ratio"  on  practise  has  a  bearing  on 
the  theory  of  this  test.  The  fact  that  the  ratio  is  high  at  the  first 
experience  in  cancelling  for  a  pair  of  digits  shows  that  the  de- 
tection of  a  pair  of  digits  in  a  group  is  a  specialized  performance, 
not  reducible  to  the  acts  of  detecting  the  single  digits.  The  de- 
tection of  any  specified  pair  of  digits  is  no  doubt  a  specialized 
performance,  susceptible  of  very  special  training;  this  has  indeed 
been  shown  in  similar  cases  by  Thorndike  and  Woodworth.  But 
in  the  present  series  of  tests,  the  pair  of  digits  cancelled  was 
changed  with  each  new  trial,  so  that  the  training  visible  in  the 
lowering  of  the  ratio  from  yy  per  cent,  to  72  or  70  per  cent,  is  an 
example  of  transferred  practise,  and  indicates  that  there  is  some 
element  of  skill  common  to  the  checking  of  all  the  pairs  of  digits. 

Though  the  ratio  varies  within  rather  narrow  limits  and  shows 
a  comparatively  small  A.D.    (as  seen   in  the   table  on   p.   36), 
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yet  there  is  sufficient  variation  to  make  it  possible  that  the  ratio 
varies  according  to  the  nature,  difficulty,  etc.,  of  the  pair  em- 
ployed in  the  particular  test.  We  have  been  unable  to  find  any 
characteristic  difference,  however,  according  to  the  difficulty 
of  the  digits  entering  into  the  pair,  except  that,  in  case  of  subject 
R.S.W.,  the  ratio  is  low  for  pairs  containing  the  digit  I.  The 
subject,  after  practise,  gave  the  following  ratio  for  pairs  contain- 
ing the  several  digits. 

Pairs  containing  the  digit               Average  ratio      P.  E.  A.  D. 

i 68.2  0.5  1.7 

2 73-1  11  3-6 

3 72.8  0.8  2.7 

4 72.8  0.8  2.8 

5 72.1  0.7  2.3 

6 73-4  0.7  2.4 

7 73.7  0.7  2.3 

8 71-8  0.9  3.1 

9 74.8  0.8  2.6 

In  general,  the  ratio  seems  not  to  depend  on  the  digit;  and 
the  same  negative  result  appears  in  case  of  the  other  subject, 
J.  W.  T.,  who  moreover  does  not  show  anything  characteristic 
of  the  digit  1.  But  the  above  results  from  subject  R.S.W.  show 
an  unmistakeably  lower  ratio  for  pairs  containing  the  digit  1. 
The  distribution  is  fairly  bimodal,  the  pairs  containing  1  forming 
a  group  by  themselves. 

Some  explanation  of  the  low  ratio  for  pairs  containing  the 
digit  1  is  afforded  by  R.S.W.'s  introspective  account.  It  early 
occurred  to  him  that  a  good  device  for  cancelling  groups  con- 
taining a  pair  would  be  to  look  first  for  the  easier  digit  of  the 
pair,  and  thus  to  look  for  the  harder  digit  only  in  the  groups 
where  the  easier  digit  appeared.  In  practise,  however,  this  device 
did  not  seem  to  him  to  work  very  well,  except  when  the  easier 
digit  was  1 ;  he  tried  to  use  the  device  also  when  the  easier  digit 
was  4,  7  or  8,  but  without  subjective  indications  of  success. 
When  one  of  the  digits  was  1,  groups  containing  it  could  be 
recognized  in  indirect  vision,  and  thus  many  groups  could  be 
passed  over  altogether  in  direct  vision.    Subjectively,  this  method 
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of  working  required  more  effort  but  appeared  successful.  The 
objective  records,  as  crystallized  in  the  "ratio,"  show  that  the 
device  was  a  success  in  the  case  of  the  digit  i. 

Further  consideration  of  this  point  may  throw  some  light  on 
the  mental  process  involved  in  this  test.  If  finding  a  pair  were 
the  same  thing  as  finding  the  members  of  the  pair,  with  no  over- 
lapping, the  time  for  the  pair  would  be  the  sum  of  the  times  for 
the  digits  composing  the  pair — instead  of  being,  on  the  average, 
only  70  per  cent  of  that  sum.  There  must  therefore  be  considerable 
overlapping  or  condensation.  On  the  motor  side,  there  is  a  pos- 
sible condensation  of  the  checking  movement,  but  this  is  so  quick 
and  automatic  anyway  that  abbreviating  it  has  probably  little  to 
do  with  the  shortening  of  the  time.  More  strain  is  probably  put 
on  the  eye  movements  when  the  speed  of  the  work  approaches  its 
maximum  (about  3  groups  covered  per  second)  ;  but  since  this 
maximum  is  not  approached,  in  our  results  so  far,  except  by  one 
subject  in  case  of  the  single  digit  1,  the  probability  is  that  the  de- 
mands made  on  the  eye  are  well  within  its  motor  capacity.  The 
difficulty  of  these  tests  is  mainly  perceptional,  and  the  over- 
lapping which  is  effective  in  finding  pairs  of  digits  must  occur  in 
the  perceptive  process. 

If  the  device  described  above  as  adopted  by  one  subject  in  find- 
ing pairs  of  digits — a  device  which  has  frequently  been  adopted 
by  other  subjects  in  similar  tests — if  this  device  represented  the 
essentials  of  finding  the  pair  quickly,  then  the  following  calcula- 
tion should  hold  good.  The  subject  looks  first  for  only  one  digit, 
and  where  he  finds  it  looks  for  the  other  one.  The  task  of  looking- 
for  the  second  digit  would  be  necessary  only  in  2/z  of  the  total 
number  of  groups  in  the  blank  (since  the  first  digit,  or  any  digit, 
is  present  in  56  out  of  the  84  groups) .  If  therefore  this  plan  were 
carried  out  systematically  and  without  hitch,  the  time  for  checking 
a  pair  should  be  equal  to  the  time  for  checking  the  first  digit  plus 
2/z  of  the  time  necessary  to  check  the  second  digit  in  the  entire 
blank.  For  example,  in  checking  the  groups  containing  both  1 
and  2,  the  time  would  be  that  needed  to  find  the  i's  (and  this  is 
31  sees.)  plus  2/z  of  the  time  necessary  to  go  through  the  blank 
for  2,  namely  2/z  of  45.5  seconds,  or  30.3  seconds ;  which  added  to 
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31.0  sees,  gives  61.3  sees,  as  the  calculated  time  for  checking  the 
pair  12.  But  the  observed  time  is  considerably  less  than  this, 
namely  53  sees. ;  and  this  same  discrepancy  between  the  calculated 
and  observed  values  obtains  in  every  instance.  The  time  for 
checking  a  pair  is  never  as  long  as  it  would  be  if  the  above 
device  were  followed  systematically. 

This  device  does  not  therefore  constitute  the  essential  mechan- 
ism of  checking  a  pair  of  digits.  The  device  seemed  to  work 
well  with  one  subject,  in  case  one  of  the  digits  was  1 ;  but  its 
conscious  use  only  reduced  the  "ratio"  from  72.5  to  68.2,  or  4 
units,  whereas  something  else  reduced  the  average  pair  from 
the  maximum  of  100  to  72.5,  or  27  points.  There  must  ac- 
cordingly be  some  form  of  overlapping  of  which  the  subject  is 
not  clearly  conscious,  but  which  is  much  more  efficacious  than 
the  best  devices  which  he  consciously  adopts.  Introspection  gives 
some  hints  at  such  an  overlapping.  Sometimes,  indeed,  a  group 
is  successively  examined  for  the  two  digits  composing  the  pair; 
but  this  is  rather  the  exception.  Often  the  presence  of  both 
digits  is  simultaneously  recognized;  and  still  oftener  the  absence 
of  the  pair  is  recognized  without  a  clear  consciousness  of  which 
of  the  two  digits  is  lacking. 

(3)  Cancelling  of  groups  containing  three  specified  digits. 
Our  data  here  are  limited  to  25  tests  with  one  subject.  The  time 
occupied  in  this  test  is,  again,  closely  correlated  with  the  sum  of 
the  times  for  the  three  component  digits,  and  is  equal  to  about 
70  per  cent,  of  the  sum  of  these  times.  Apparently  the  ratio  is 
slightly  lower  for  three  digits  than  for  a  pair,  for  the  subject, 
R.S.W.,  gives  an  average  ratio  of  68.3  per  cent.,  with  A.D.  of 
5.4,  and  P.E.  of  the  average  of  0.9.  This  average  is  thus  prob- 
ably lower  than  the  average  of  72.5  obtained  by  this  subject  with 
pairs. 

When  1  is  one  of  the  three  digits,  the  average  ratio  is  62.0, 
A.D.  being  3.4  and  P.E.  1.1.  The  subject  adopted  the  same 
device  as  in  pairs  containing  the  digit  1,  and  the  results  here  are 
confirmatory  of  what  has  previously  been  said. 

For  two  approximately  equivalent  tests,  we  recommend  can- 
celling for  146  and  for  257.    The  time  for  each  of  these  is  about 
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2.5-3  times  that  for  the  single  digit  1,  or  about  2-2.5  times  that 
for  the  single  digit  4  or  7. 

Use  of  the  number-checking  blanks  with  laboratory  classes. 
As  suggested  above  (p.  27),  the  number-checking  blanks  could 
readily  be  adapted  for  experiments  in  continued  mental  work,  in- 
terference, etc.  One  of  the  writers  has  used  Form  B,  the  "Num- 
ber-group Blank,"  with  success  in  an  experiment  in  practise  and 
"transfer."  Half  of  the  blank  being  used  as  the  unit,  the  sub- 
ject first  checked  the  groups  containing  the  digit  6,  then  took  a 
practise  series  of  ten  units  with  the  digit  7;  then  one  unit  again 
with  6;  then  another  practise  series  with  digit  4;  then  one  unit 
with  7  and  finally  one  more  with  6.  The  two  methods  of  studying 
"transfer,"  namely  the  "cross-section"  method,  and  what  may  be 
called  the  "successive  practise  curve"  method,1  are  combined  in 
this  experiment.  The  tests  with  digit  6  give  cross-sections  before 
and  after  practise  with  other  digits;  and  since  the  digits  4  and 
7  are  equally  hard  to  find,  the  practise  curve  with  the  one,  follow- 
ing that  with  the  other,  should  show  the  effects  of  the  preceding 
practise.  Transfer  is  pretty  sure  to  be  in  evidence  in  each  stu- 
dent's results;  these  need,  to  be  sure,  some  correction  from  con- 
trol experiments  in  which  the  cross-sections  are  taken  without  the 
intervening  practice.2 

1  Introduced  by  Bair,  'The   Practice  Curve",  Psychol.  Rev.,  Monogr.  Suppl. 
No.   19,   1902. 
3  See  W.  F.   Dearborn,  Psychol.  Bulletin,  190Q,  6,  44. 
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For  rigidity  of  associative  control,  no  experiments  surpass 
those  involving  the  simple  arithmetical  processes.  In  these  a  cer- 
tain arithmetical  task  is  visually  presented  to  the  subject,  and 
efficiency  is  measured  in  terms  of  time  and  error.  To  provide 
an  objective  criterion  of  the  performance  the  subject  is  required 
to  speak  or  write  the  result.  According  to  what  has  gone  be- 
fore, oral  response  is  employed  exclusively  in  the  present  experi- 
ments. One  advantage  of  the  written  response  is  thus  dispensed 
with,  namely  the  permanency  of  the  record,  through  which  to 
check  its  accuracy.  This  difficulty  is  best  obviated  through  pro- 
viding the  operator  with  a  key  upon  which  the  correct  reactions 
are  noted.  The  operator  follows  the  responses  of  the  subject  on 
the  key  and  so  keeps  account  of  the  data  to  be  recorded. 

Such  experiments  with  arithmetical  processes  have  an  almost 
infinite  range  of  difficulty,  varying  in  practise  from  the  simple 
addition  of  a  pair  of  digits  to  the  mental  multiplication  of  three 
and  even  four  place  numbers.  The  chief  advantage  of  the  former 
is  their  freedom  from  errors ;  of  the  latter,  the  greater  proportion 
of  time  spent  in  the  essential  work  of  the  test.  At  first  glance, 
one  might  consider  that  this  same  consideration,  which  leads  to 
the  substitution  of  oral  for  written  response,  should  lead  to 
the  rejection  of  the  easier  and  adoption  of  the  more  difficult 
experimental  material.  But  it  were  very  easy  to  press  this  ad- 
vantage too  far,  especially  in  tests  that  are  intended  for  any- 
thing like  general  employment.  The  more  complex  intellectual 
associations  would  result  in  the  average  individual  in  an  im- 
possible number  of  errors,  if  indeed  they  did  not  prove  too  much 
for  his  patience  as  well  as  his  powers.  A  test  not  intended  for 
limited  application  should  not  be  one  limiting  the  subjects  who 
can  respond  to  it;  the  tests  to  be  described  here,  therefore,  deal 
with  the  simpler  arithmetical  processes,  regularly  of  addition, 
though  the  material  prepared  is  adaptable  in  various  ways. 

i.     The  particular  form  of  addition  test  with  which  the  most 
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work  has  been  done,  and  whose  properties  with  reference  to  the 
work  curve  are  best  understood,  is  that  of  the  Kraepelinian 
Rechenhefte.  This  is  a  pamphlet  of  twenty-four  pages,  upon  each 
of  which  are  printed  nine  vertical  columns  of  32  single  digits  in 
apparently  random  succession.  It  is  possible  to  experiment  with 
this  material  by  continuously  adding  the  successive  digits,  and 
announcing  the  sum  total  at  stated  points.  The  disadvantages  o* 
this  procedure  are  very  numerous  and  do  not  call  for  considera- 
tion here.  A  decidedly  preferable  method  is  the  simple  addition 
of  the  successive  pairs  of  digits.  That  is,  the  first  four  figures  in 
the  Rechenheft  being  8,  3,  5,  7  the  sums  announced  by  the  sub- 
ject are  11,  8  and  12.  The  subject  continues  to  announce  the 
sum  of  every  figure  plus  the  one  next  below  it.  Precise  control 
of  the  whole  process,  both  as  to  accuracy  and  time,  is  thus  secured. 
As  before  mentioned,  the  operator  checks  the  correctness  of  the 
sums,  notes  errors,  and  the  amount  performed  within  specified 
times.  The  usual  periods  of  work  with  the  Kraepelinian  test 
have  been  of  five  and  ten  minutes  each,  and  it  has  also  been 
customary  to  record  the  amount  of  work  done  during  the  single 
minutes.  The  subject  should  not,  as  has  been  done,  be  called  upon 
to  make  the  records;  all  such  tasks  devolve  properly  upon  the 
experimenter.  It  does  not  appear  that  a  significant  portion  of 
the  time  is  consumed  in  the  motor  process  of  response.  At  the 
beginning  of  practise,  the  number  of  additions  made  in  five  min- 
utes is  usually  under  two  hundred.  It  need  scarcely  be  said  that 
the  sums  themselves  could  be  read  in  a  much  shorter  time ;  maxi- 
mum speed  of  reading  aloud  in  normal  individuals  averages  not 
far  from  100  words  in  30  seconds. 

Used  in  the  above  way  the  Kraepelin  Rechenheft  contains  31 
additions  per  column,  279  per  page.  While  the  unpractised  sub- 
ject is  not  likely  to  do  more  than  this  in  five  minutes,  a  little 
practise  will  soon  take  him  over  the  page,  and  it  may  be  con- 
sidered always  advisable  to  open  the  Rechenheft  to  two  full  pages, 
fit  should  be  held  open  with  a  clip,  amply  providing  for  any  com- 
mon performance  without  turning  a  leaf.  One  of  our  subjects, 
after  prolonged  practise,  occasionally  reached  a  figure  above  558 
additions  in  the  five  minute  period.     There  is  uniformly  a  con- 
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siderable  practise  effect  in  the  test,  perhaps  due  partly  to  the 
unfamiliar  sort  of  reaction  required  to  the  material,  but,  as  with 
the  number-checking  test,  the  individual  differences  in  suscepti- 
bility to  practise  are  small  in  comparison  to  the  differences  in  the 
amount  of  work  performed. 

The  Kraepelin  Rechenheft  is  practically  the  only  form  of  this 
experiment  that  permits  long  continued  tests  of  the  same  subject 
with  sufficient  uniformity  of  experimental  material;  but  as  the 
present  tests  are  not  designed  as  practise  experiments,  it  was 
thought  advisable  to  construct  a  more  convenient  blank  for  a 
single  or  small  number  of  determinations.  It  was  also  endeavored 
to  improve  on  certain  minor  features  of  the  Rechenhefte,  as  the 
odd  number  of  columns  and  additions  in  each  column. 

The  blank,  reproduced  on  the  page  opposite,  contains  24 
columns  of  26  digits  (25  additions), — in  all,  600  additions.  This 
is  probably  beyond  the  5 -minute  capacity  of  the  normal  subject  at 
the  limit  of  practise.  The  columns  are  arranged  in  six  groups 
of  four,  each  thus  containing  100  additions  with  104  digits.  The 
scheme  of  the  distribution  of  the  digits  was  simpler  than  in  the 
number-checking  test.  99  slips  were  prepared,  eleven  for 
each  of  the  nine  digits.  The  five  remaining  slips  bore  the  digit  5. 
Random  drawings  were  made  from  the  group  of  slips,  and  the 
digits  were  written  in  the  columns  in  the  order  in  which  they 
were  drawn.  Each  set  of  four  columns  (100  additions)  is  there- 
fore a  unit  in  itself,  and  is  made  up  of  a  proportionate  number 
of  each  of  the  nine  digits  in  random  order.  The  six  series  of 
100  additions  may  then  be  considered  as  homogeneous  and  of 
approximately  equal  difficulty. 

Subjects  are  not  apt  to  comprehend  this  test  so  readily  as  they 
do,  for  example,  the  number-checking  test.  They  may  tend  to 
add  the  numbers  continuously,  or  more  especially  to  add  discrete 
numbers,  thus  with  the  column  beginning  as  before,  8,  3,  5,  7,  to 
announce  the  sums  11  and  12,  omitting  to  add  the  3  and  5.  The 
operator  must  be  well  assured  from  the  instructional  material  that 
the  test  is  properly  comprehended.  Some  subjects  prefer  to  fol- 
low the  columns  with  the  finger ;  this  should  be  permitted,  on  the 
same  principle  that  permits  reversed  directions  in  the  number- 
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checking  test.  The  addition  of  the  final  digit  of  one  column  to  the 
beginning  one  of  the  next  is  not  required,  nor  do  subjects  ordin- 
arily tend  to  do  it,  even  though  not  specially  instructed. 

The  test  may  of  course  be  made  of  any  length  within  the  limits 
of  the  blank,  but,  from  experience  with  the  Rechenhefte  one  hun- 
dred additions  appear  sufficient  for  a  unitary  determination. 
The  length  of  the  present  blank  is  believed  to  provide  amply  for 
sufficient  variation  of  the  experimental  material  for  all  ordinary 
purposes  outside  of  special  research  upon  the  individual  function. 

The  test  requires  the  constant  attention  of  the  operator  to  the 
key  to  check  the  proper  performance  of  the  work  by  the  sub- 
ject. False  reactions  are  more  frequent  than  in  the  number- 
checking  test,  and  they  should  be  kept  track  of,  though  the  writer 
has  not  seen  an  instance  where  they  obscured  individual  differ- 
ences in  efficiency.  The  subject  may  not  notice  the  error;  if  he 
does,  the  purpose  should  be  to  get  him  over  it  with  as  little  dis- 
turbance and  distraction  as  possible,  and  he  should  therefore  be 
allowed  to  correct  it  or  not,  whatever  is  the  path  of  least  resis- 
tance for  him.  We  believe  this  to  be  the  sounder  experimental 
practise,  whatever  might  be  said  of  it  from  an  ethical  standpoint. 

2.  In  order  to  furnish  a  regulated  experimental  material 
which  should  have  a  greater  flexibility  of  application  than  is 
usual  in  this  class  of  tests,  a  second  form  of  procedure  is  sub- 
mitted, known  as  the  constant  increment  test.  This  is  a  little- 
recognized  inethod,  but  one  which  in  direct  comparison  has  shown 
superiority  over  other  forms.  It  consists  in  presenting  to  the 
subject  a  series  of  numbers,  requiring  the  identical  arithmetical 
operation  to  be  performed  upon  each.  In  the  observations  made 
with  this  test,  the  usual  procedure  has  been  the  addition  of  4. 
In  this  particular  instance,  there  is  perhaps  no  reason  why  the 
Kraepelin  blank  should  not  serve  this  purpose  as  well  as  that  for 
which  it  is  ordinarily  used.  In  order  however,  to  make  the  ma- 
terial adaptable  also  to  subtraction,  especially  of  larger  figures,  it 
is  thought  wise  to  preserve  the  special  blank  originally  adapted 
to  this  test. 

This  blank  contains  100  two  place  numbers.  The  unit  places  in 
these  numbers  are  ten  each  of  the  figures  1,2,3,4,5,6,7,8,9,0.     In 
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the  tens  places  are  ten  2's,  ten  y's  and  twenty  each  of  the  interven- 
ing 3,4,5  and  6.  These  features  are  symmetrically  distributed  with 
reference  to  the  halves.  The  subject  speaks  the  proper  responses 
according  to  the  assigned  experimental  task ;  thus  in  the  addition 
of  4  the  responses  begin  68,  53,  66,  etc.  It  is  probably  advisable 
to  confine  the  unit  of  observation  to  half  of  the  blank,  and  even 
better  would  probably  be  two  tests  of  one  column  each.  Errors 
seem  to  be  but  slightly  more  frequent  than  in  the  simple  addition 
of  a  pair  of  numbers ;  in  their  treatment,  the  same  considerations 
obtain  as  in  the  previous  form  of  test.  Since  the  operation  to  be 
performed  with  the  given  numerals  may  vary  indefinitely,  no 
key  is  provided,  but  the  experimenter  may  readily  provide  one 
himself  for  his  own  particular  requirements,  and  should  always 
do  so;  its  employment  being  the  same  as  in  the  Kraepelin  form 
of  addition  test. 

Results.  Individual  differences,  due  in  part,  no  doubt,  to  dif- 
ferences of  training,  are  very  great  in  even  the  simplest  arithme- 
tical tests.  Thus,  while  one  of  the  authors  has  usually  obtained, 
with  the  Kraepelin  form  of  test,  times  of  from  two  to  three 
minutes  for  100  additions,  the  other  of  us,  working  on  7  college 
and  university  students,  has  the  following  results : 

Average  time  for  100  additions 107.2  seconds 

A.  D 24.4  seconds 

Range    ; 65-164  seconds 

With  the  constant  increment  test,  the  following  results  have 
been  obtained  from  10  subjects  of  the  same  class  as  above :  Only 
one  column  was  used  in  each  test,  and  the  times  given  are  times 
for  one  column. 

Problem Add  4 

Av 33-9 

A.  D 5-8 

Range    24-49 

Av.  errors  per  column  of  2S .  .  0.3  0.2  2.4 

Experience  with  other  subjects  leads  one  to  suspect  that  the 
time  per  column,  for  adding  4,  will  often  run  up  above  60  seconds. 


Subtract  4 

Add  17 

41. 1 

97-4 

11.0 

23.6 

25-67 

62-158 

V.     NAMING  TESTS 

i.  The  Color  Naming  Test.  No  doubt  the  best-trained  and 
quickest  of  all  associative  responses  appear  in  the  reading  of 
words,  letters  and  numbers.  (See  above  p.  9).  Next  to  this, 
probably,  stands  the  naming  of  familiar  objects.  The  color- 
naming  test  belongs  here,  and  may  be  expected  to  give  shorter 
times,  per  reaction,  than  any  of  the  other  tests  included  in  the 
present  selection. 

The  test  blank  shows  100  patches  of  color  (besides  5  sample 
patches),  each  1  cm.  square  and  separated  by  spaces  of  1  cm. 
from  its  neighbors.  The  colors  are  to  be  named  in  order,  as  in 
reading,  and  one  side  is  indicated  as  the  top.  The  arrangement 
is  such,  however,  that  the  blank  can  be  equally  well  used  in  any 
one  of  the  four  positions. 

Care  is  needed  in  the  selection  of  colors  for  such  a  test;  for 
the  color  names  required  should  be  of  universal  familiarity,  and 
there  are  few  such  names.  White,  black,  gray,  red,  yellow,  green, 
blue,  brown,  and  possibly  pink  and  purple  are  in  sufficiently 
general  use  for  our  purpose.  It  seemed  safer  to  exclude  pink  and 
purple;  if  white  is  then  used  for  the  background,  seven  colors 
remain;  but  it  seemed  better,  on  the  whole,  to  employ  only  five 
stimulus  colors,  repeating  each  twenty  times.  After  some  experi- 
menting, it  was  decided  to  use  red,  yellow,  green,  blue  and  black. 

The  use  of  the  Hering  red  and  green  seemed  undesirable, 
since  they  do  not  suggest  the  color  names  so  promptly  as  do  a 
red  and  green  each  somewhat  nearer  to  yellow.  To  attempt  to 
equate  the  colors  in  brightness  would  not  be  best,  since  this  would 
mean  toning  down  the  yellow  to  a  point  at  which  it  would  be 
likely  not  to  be  called  by  that  name.  The  blue  must  be  pale  in 
order  to  be  distinguished  from  black  as  quickly  as  the  other  colors 
are  distinguished  from  each  other. 

In  the  arrangement  of  the  color  stimuli  on  the  blank,  all 
sequences  of  the  same  color  were  avoided,  as  such  direct  sequences 
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enable  the  subject  to  combine  two  reactions,  practically,  into  one. 
Too  frequent  recurrence  of  the  same  sequence  of  different  colors 
was  also  avoided,  the  object  being  to  compel  a  separate  reaction 
to  each  single  stimulus.  It  was  desired,  indeed,  to  have  the  dif- 
ferent possible  sequences  of  two  and  of  three  colors  occur  with 
equal  frequency ;  but  something  less  than  perfection  in  this  regard 
had  to  be  accepted,  for  it  was  desired  also  to  make  the  arrange- 
ment equally  good  for  each  of  four  positions  of  the  blank;  and 
all  the  conditions  of  perfection  could  not  be  met  in  both  the 
horizontal  and  vertical  lines  at  the  same  time.  An  incredible 
amount  of  time  was  consumed  in  arranging  the  colors  to  meet 
these  simple  requirements.  The  blank  seems  now  to  be  free 
from  serious  blemishes  of  arrangement,  and  requires  approxi- 
mately the  same  time  for  reading  in  each  of  the  four  positions. 

Preparatory  to  the  test,  the  experimenter  lays  the  blank  before 
the  subject,  with  only  the  sample  line  of  5  colors  showing.  The 
subject  is  directed  to  give  the  names  of  the  sample  colors;  when 
he  understands  the  task  and  knows  what  names  to  use,  the  whole 
blank  is  exposed  at  the  word  "Go!"  The  time  for  the  first 
half,  as  well  as  for  the  whole  blank,  may  well  be  taken.  Re- 
peated trials  can  be  made  with  the  blank  in  four  positions.  The 
line  of  5  sample  colors  is  to  be  omitted  in  the  actual  test.  As 
usual,  one  half  of  the  blank  is  long  enough  for  a  satisfactory 
test.  A  key  of  the  series  of  correct  responses  will  facilitate  the 
experimenter's  task. 

2.  The  Form  Naming  Test.  The  blank  next  to  be  described 
under  the  head  of  the  Substitution  Test  is  similar  in  all  respects 
to  the  color  sheet,  except  that  five  geometrical  figures  take  the 
place  of  the  five  colors.  This  blank  can  be  used  in  the  same 
manner  as  the  color  blank,  the  numbers  written  in  the  key  line 
being  disregarded. 

Results.  Here,  as  in  most  of  the  other  tests,  the  results  now 
available  are  insufficient  to  do  more  than  give  a  general  impression 
of  the  time  required.  In  this  case,  too,  the  results  are  all  from 
one  class  of  subjects,  namely  college  and  graduate  students.  The 
whole  blank  of  100  stimuli  was  reacted  to,  and  the  time  taken  for 
the  half  as  well  as  for  the  whole. 


NAMING  TESTS 
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Color  naming  test. 

1st  half 

9  men       Av 30.6 

A.   D 3-0 

P.  E 0.8 

5  women  Av 26.4 

A.  D 4-5 

P.  E 1.7 

Total  range,  both  sexes  together 22-41 

Form  naming  test. 

1st  half 

6  men       Av 46.7 

A.  D 8.6 

P.    E 2.9 

4  women  Av 38.5 

A.  D 6.0 

P.  E . .  2.5 

Total    range,    both    sexes    together....  31-60 


2nd  half 

Whole 

35.1 

64.6 

5-2 

7-4 

1-5 

2.0 

29.0 

55-4 

3-6 

8.1 

1.3 

3.0 

24-48 

48-89 

2nd  half 

Whole 

47.2 

93-8 

7.2 

15-5 

2.4 

5-2 

42.3 

80.8 

10.0 

16.0 

4.2 

6.7 

29-58 

60-117 

From  these  data,  it  may  be  inferred  (1)  with  reasonable  as- 
surance, that  the  color  naming  test  is  easier  than  the  form-nam- 
ing test.  Comparison  of  the  figures  for  the  present  color-nam- 
ing test  with  those  reported  by  Wissler1  from  the  same  class  of 
subjects  with  the  use  of  the  Columbia  color-naming  test,  which 
employs  ten  colors,  makes  it  probable  that  the  present  test  with 
five  colors  is  noticeably  easier — as  was,  indeed,  intended. 

(2)  It  may  be  inferred  from  the  above  table,  with  much 
probability,  that  a  sex  difference  exists  in  the  case  of  the  color 
naming  test,  women  being  on  the  average  quicker  than  men. 
This  is  the  more  probable  because  Wissler2  obtained  the  same 
sex  difference  from  much  more  extensive  data. 

(3)  It  seems  also  probable  that  the  same  sex  difference  exists 
in  the  case  of  the  form-naming  test.  If  so,  the  sex  difference 
here  in  question  is  not  specially  related  to  the  color  sense,  but 
rather  to  linguistic  facility.  The  authors  have  in  mind  the  ac- 
cumulation of  sufficient  data  to  determine  whether  the  appear- 
ance here  shown  corresponds  to  a  real  sex  difference. 

(4)  In  the  color-naming  test  it  seems  probable,  and  in  the 

1  Psychol.  Rev.,  Monograph  Suppl.  No.  16,  1901. 
1  Op.  cit. 
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form-naming  test,  possible,  that  there  is  some  slackening  of  re- 
actions, such  as  was  shown  above  (p.  15)  to  occur  in  other  forms 
of  test.  On  reference  to  the  individual  records,  we  find  that  10 
of  the  14  subjects  in  the  color-naming  test  took  longer  for  the 
second  half  than  for  the  first  half  of  the  test;  and  in  the  form 
naming  test,  6  out  of  10  did  the  same.  Only  one  of  the  ex- 
ceptions is  more  than  a  bare  exception.  The  behavior  of  the 
subjects  during  the  test  shows  periods  of  hesitation  and  ob- 
struction, and  even  of  false  reaction —  rather  a  strange  phenome- 
non, in  view  of  the  great  familiarity  of  the  names  and  their 
correct  use  immediately  before.  The  subject  is  aware  of  this 
inhibition,  and  it  is  a  strange  experience  for  him.  The  "mechan- 
ism" of  inhibition  can  not  here  have  the  elaborate  Freudian  char- 
acter ;  and  in  fact  the  experiment  seems  a  good  one  to  show  the 
reality  of  other  forms  of  inhibition  in  the  recall  of  names.  The 
real  mechanism  here  may  very  well  be  the  mutual  interference 
of  the  five  names,  all  of  which,  from  immediately  preceding  use, 
are  "on  the  tip  of  the  tongue",  all  equally  ready  and  therefore 
likely  to  get  in  one  another's  way.  These  periods  of  inhibition 
do  not  appear  at  the  very  beginning  of  the  test,  but  most  often, 
to  judge  from  incidental  observation,  along  in  the  middle.  Some 
subjects,  after  succumbing  a  few  times  to  interference,  appear 
to  collect  themselves  and  do  the  last  part  of  the  test  better  than 
the  middle.  We  have  the  records  of  the  successive  rows  of  ten 
stimuli  each,  in  the  case  of  five  subjects  in  each  test.  The  aver- 
age time  in  seconds  for  each  row  is  as  follows. 

Row   1        2        3        4        5        6        7        8        9       10 

Color  naming  ...  6.2  5.6  5.6  6.6  6.8  7.3  7.7  6.9  6.3  6.2 
Form  naming  ...     8.0      9.1       9.0     10.3     10.8       8.2     10.5       9.9     10.3       9.0 


VI.     FORMATION  OF  NEW  ASSOCIATIONS 

The  Substitution  Test.  This  blank  is  modelled  after  one  by 
Professor  J.  E.  Lough,1  but  is  simplified,  in  that  only  five  (instead 
of  twenty)  different  stimuli  are  used;  at  the  same  time,  by  em- 
ploying geometrical  forms  in  place  of  the  letters  of  Professor 
Lough,  it  is  partly  freed  from  the  danger  that  some  subjects  may 
hit  upon  easy  mnemonics. 

Since  the  names  of  the  forms  may  enter  into  the  subject's  pro- 
cedure, the  forms  should  have  equally  familiar  names.  They 
should  also  be  of  such  shapes  that  the  blank,  like  the  color-nam- 
ing blank,  may  be  capable  of  use  in  different  positions.  Only 
about  five  geometrical  forms  meet  these  conditions :  the  circle, 
square,  triangle,  star  and  cross.  The  blank  is  made  up  of  these 
fi\Q  forms,  each  repeated  twenty  times.  The  arrangement  of 
the  stimuli  follows  the  same  rule  here  as  in  the  color-naming 
test. 

At  the  top  of  the  blank  appears  a  line  containing  each  of  the 
five,  forms  once,  with  a  number  on  each.  This  line  being  cover- 
ed, the  rest  of  the  blank  is  exposed  to  the  subject,  and  it  is  ex- 
plained to  him  that  he  is  to  write  on  each  of  these  forms  a  num- 
ber,—  the  same  number  as  he  will  find  on  that  form  in  the  key 
at  the  top.  In  this  test,  the  general  rules  of  instructing  the  sub- 
ject by  aid  of  examples  can  not  be  exactly  followed;  for  the  as- 
sociation to  be  employed  in  the  test  should  not  be  formed  before 
the  outset  of  the  test  itself,  since  this  is  a  test  of  the  formation  of 
associations.  When  the  experimenter  is  sure  that  the  subject 
understands  what  is  to  be  done,  he  uncovers  the  key,  at  the  word 
"Go!"  Besides  taking  the  time  for  the  whole  and  half,  the  ex- 
perimenter may  be  able  to  get  the  times  for  each  successive  line, 
and  so  obtain  a  curve  of  the  formation  of  the  associations. 

A  misunderstanding  which  has  occasionally  appeared  in  the 
use  of  this  test  should  be  guarded  against  in  the  instructions  to 
the  subject.  Some  subjects  have  started  to  go  through  the 
blank  numbering  only  one  of  the  forms  at  a  time,  intending  to 

1  Described  by  Kirkpatrick,  Studies  in  Development  and  Learning,  Arch,  of 
Psychol.,  1909,  No.  12,  p.  36. 
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go  through  again  for  each  of  the  other  forms.  It  should  be 
made  clear  that  the  forms  are  to  be  numbered  in  order  as  in 
reading  or  writing. 

Results.  Eleven  educated  adults  (6  men,  5  women)  gave  the 
following  average  time  in  seconds : 

1st  half 

Av 79.6 

A.  D J 9.0 

P.  E 2.3 

Range 58-94 

The  gain  from  the  first  half  to  the  second  is  perhaps  not  so 
great  as  would  have  been  expected.  In  fact,  few  if  any  of  the 
subjects  fully  mastered  the  key  in  the  course  of  the  100  reactions. 

Time  was  taken,  in  this  test,  for  each  successive  row  of  ten 
forms,  with  the  following  average  results  (11  subjects)  : 

Row   1        2        3        4        5        6 

Av 15. 1    17.7    16.4    15.6    14.8    13.8 

A.  D 3.7      2.8      2.8      1.6      1.6      2.3 

The  longer  time  for  the  first  line  than  for  the  second  is  found 
in  9  of  the  11  individuals;  and  3  subjects  do  the  first  line  as 
quickly  as  any  other.  It  is,  in  short,  possible  to  do  the  first  line, 
by  simple  use  of  the  key,  in  9-1 1  seconds,  and  this  is  as  rapid  as 
any  of  the  subjects  became  during  the  course  of  the  test.  Some 
subjects  do  the  first  line  by  mere  copying  from  the  key;  others 
start  to  memorize  and  take  longer  on  this  line;  this  is  probably 
the  cause  of  the  extra  large  variability  for  the  first  line. 

The  test  is  not  long  enough  to  permit  the  complete  establish- 
ment of  the  associations;  several  blanks  may  be  used  in  succes- 
sion, and  oral  may  be  substituted  for  written  responses  in  order 
to  simplify  the  motor  part  of  the  performance.  At  the  best,  how- 
ever, progress  is  rather  slow ;  and,  indeed,  one  would  not  expect 
these  freshly  formed  associations  to  surpass  readily  the  familiar 
associations  involved  in  the  form-naming  test  (p.  51),  the  times  in 
which  are,  after  all,  not  very  much  shorter  than  those  in  the 
last  rows  of  the  substitution  test. 
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By  use  of  a  second  key,  the  blank  can  be  used  for  the  study 
of  interference;  one  of  the  authors  has  so  used  it  in  laboratory 
classes,  following  the  general  arrangement  of  Bergstrom's  card- 
sorting  experiment.1  The  blank  can  also  be  used  for  a  simple 
cancellation  test,  similar  to  the  number-checking  tests. 

1  Amer.  Journ.  of  Psychol.,  1893,  5,  356. 
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VII.     LOGICAL  RELATIONS 

The  form  of  test  in  which  the  stimulus  is  a  word  and  the 
response  another  word  standing  in  some  assigned  logical  relation 
to  the  stimulus  has  been  long  and  widely  used,  and  has  an  intellec- 
tual atmosphere  that  makes  it  seem  likely  to  prove  a  test  of  indi- 
vidual differences  of  the  intellectual  sort.  At  the  same  time,  it  is 
distinctly  a  test  of  the  command  of  language,  and  when  the 
measurement  concerns  the  speed  of  the  response,  familiarity  with 
the  necessary  words  is  a  prime  necessity.  Unless  great  care  is 
used  in  the  selection  of  stimulus  words,  long  reaction  times  will 
occur  from  the  need  of  searching  for  the  proper  response  words, 
and  the  test  thus  becomes  predominantly  linguistic  in  nature. 
Linguistic  it  must  always  remain  to  a  considerable  extent,  no 
matter  how  much  care  is  taken  in  the  selection  of  the  stimuli ;  but 
the  effort  should  be  to  minimize  the  linguistic  factor  by  selecting 
only  stimulus  words  that  are  universally  familiar. 

Besides  the  familiarity  of  the  associations  employed,  the  test 
calls  for  skill  in  the  handling  of  these  associations;  and  it  is  this 
skill,  most  of  all,  which  the  test  designs  to  measure.  In  other 
words,  it  is  the  efficiency  of  the  "determining  tendency",  or  ad- 
justment to  react  according  to  instructions,  which  should  be  re- 
vealed by  the  speed  of  performance.  The  more  completely  this 
adjustment  dominates  the  performance,  facilitating  the  right  re- 
sponses and  inhibiting  other,  interfering  associations  and  per- 
severations, the  less  hesitation  and  confusion  will  occur  and  the 
more  prompt  will  be  the  reaction. 

In  order  to  afford  sufficient  opportunity  for  the  determining 
tendency  or  adjustment  to  show  its  efficiency,  it  is  customary  and 
evidently  desirable  to  provide  a  number  of  stimuli  in  succession, 
requiring  the  same  sort  of  response  to  each.  There  should 
therefore  be  a  list  of  stimulus  words  for  each  of  the  logical  re- 
lations along  which  the  reactions  are  to  be  required  Thus  the 
task  of  providing  material  for  these  tests  consists  in  discovering 
a  sufficient  number  of  stimulus  words  of  the  requisite  familiarity. 
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Only  by  actual  trial  can  the  suitability  of  the  stimuli  be  ascer- 
tained. A  word  of  apparently  eminent  fitness  may  prove  to  be 
unfamiliar  to  many  subjects.  For  example,  the  word  "false" 
seemed  likely  to  be  a  good  stimulus  when  the  required  response 
was  a  word  of  opposite  meaning ;  but  in  practise  much  hesitation 
and  uncertainty  of  reaction  appeared  in  the  responses  to  this  word 
(instead  of  "true",  some  women  subjects  said  "natural").  A 
word  which  seems  perfectly  familiar  to  the  investigator  may  be 
unfamiliar  to  many  subjects,  and  a  word  which  seems  perfectly 
unambiguous  may  convey  an  unexpected  meaning  to  some  sub- 
jects. To  avoid  all  such  difficulties  with  all  subjects  is  too  much 
to  hope;  but  the  test  material  should  be  freed  from  words  that 
cause  difficulty  to  a  large  share  of  the  subjects.  This  is  necessary 
at  any  rate  if  the  time  is  to  be  taken,  not  for  each  separate  re- 
sponse, but  only  for  the  whole  series  of  responses  to  the  list  of 
stimuli ;  for  otherwise  the  total  time  may  be  determined  mostly  by 
the  difficulty  of  one  or  two  of  the  reactions.  Even  if  the  times  of 
the  separate  reactions  are  taken,  the  average  time  will  suffer  in 
the  same  way  as  the  total  time  in  the  preceding  case.  The  median 
time  is  mostly  free  from  this  source  of  error.  But  even  so,  lists 
of  nearly  uniform  difficulty  would  form  the  best  and  fairest  test 
material. 

Our  procedure  in  selecting  stimuli  for  this  class  of  tests  was 
to  start  by  getting  together  as  large  a  number  of  stimuli  as  possi- 
ble; to  eliminate  at  once  all  that  seemed  ambiguous  or  unduly 
difficult  and  to  try  the  remainder  with  a  few  subjects,  timing  the 
separate  reactions,  and  eliminating  the  stimuli  that  gave  the  slow- 
est reactions  or  that  proved  to  be  ambiguous  or  complex-arousing. 
The  abbreviated  list  was  tried  in  the  same  way  with  other  sub- 
jects, and  more  words  eliminated,  till  finally  it  appeared  that  the 
easiest  possible  list  of  stimuli  had  been  secured.  Unless  it  proved 
possible  to  secure  a  list  of  twenty  easy  stimuli,  that  particular 
test  was  abandoned.  Thus,  it  seemed  impossible  to  prepare  a  list 
of  twenty  words  sufficiently  easy  for  a  synonyms  test,  except  in- 
deed for  well-educated  individuals.  On  the  other  hand,  it  ap- 
peared possible  to  select  two  lists  of  twenty  for  the  opposites 
test. 
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The  number  of  subjects  employed  in  reaching  the  selection  of 
stimulus  words  was  greater  in  case  of  the  opposites  test  than  in 
the  others,  and  this  test  may  properly  be  regarded  as  more  highly 
"standardized"  than  any  other  belonging  under  the  head  of  logical 
relations.  In  the  case  of  opposites,  a  long  list  of  words  was  tried 
with  6  subjects,  and  the  forty  words  selected  from  this  trial 
were  tried  with  40  other  subjects;  some  need  of  revision  was 
then  apparent,  and  a  few  more  words  were  substituted  from 
tests  of  a  few  subjects;  then  the  revised  collection  of  40  was 
tried  with  thirteen  fresh  subjects,  and  a  few  minor  corrections 
still  introduced,  which  left  the  lists  in  their  present  condition. 
In  the  other  tests,  two  to  three  times  the  desired  number  of  stim- 
ulus words  were  tried  with  nine  subjects,  and  the  resulting  selec- 
tion was*  tried  with  thirteen  fresh  subjects ;  some  minor  changes 
were  then  introduced  and  the  lists  left  in  their  present  condition. 
The  "mixed  relations"  test  was  selected  gradually  on  the  basis 
of  results  from  fourteen  subjects.  % 

After  the  selection  of  the  stimulus  words  came  the  question 
of  their  arrangement  within  the  list.  This  matter  of  order  of 
stimuli  is  not  of  great  importance  if  the  time  is  to  be  taken  for 
the  separate  responses;  but  whenever  the  time  is  taken  only  for 
the  series,  the  order  of  stimuli  is  a  matter  of  some  consequence. 
We  recommend,  it  may  be  remembered,  that  the  time  be  taken 
for  the  first  half  of  the  list  as  well  as  for  the  whole  list,  and  even 
that  the  halves  be  given  as  separate  tests ;  it  is  therefore  important 
to  have  halves  of  equal  difficulty.  Moreover,  many  investigators 
find  it  convenient  to  allow  a  fixed  time  for  each  test,  and  to 
measure  the  number  of  responses  that  can  be  given  in  this  time 
("time  limit  method")  ;  with  this  procedure  it  is  important  that 
the  list  shall  be  of  uniform  difficulty  throughout,  so  that  the 
number  completed  shall  be  a  fair  measure  of  the  work  done. 
Whatever  be  the  procedure  in  giving  the  test,  the  most  desirable 
arrangement  of  the  stimulus  words  would  be  such  as  to  distribute 
the  difficulties  evenly  throughout  the  list.  If  it  were  really  pos- 
sible to  discover  twenty  stimulus  words  of  equal  difficulty,  the 
question  of  their  arrangement  would  not  arise;  but  this  is  not 
possible,  for  though  the  twenty  stimuli  be  all  decidedly  easy,  yet 
the  reaction  time  to  one  will  be  two  or  three  times  as  long  as  to 
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another  of  the  twenty,  on  the  average  of  as  many  as  ten  sub- 
jects. Since  it  is  impossible  to  prepare  a  list  of  twenty  stimuli  of 
equal  difficulty,  we  combined  the  words  in  pairs  so  that  the  pairs 
should  be  of  equal  difficulty,  as  judged  by  the  sum  of  the  re- 
action times  to  the  two  members  of  each  pair.  One  pair  thus 
may  consist  of  the  hardest  and  the  easiest  word  in  the  list,  and 
another  pair  of  two  words  of  medium  difficulty;  but  the  sum 
of  the  reaction  times  for  the  first  pair  is  equal  to  that  for  the 
second  pair,  as  judged  from  the  records  already  in  hand.  (The 
pairs  can  not  be  hoped  to  be  equal  for  all  subjects.)  These 
equal  pairs  can  then  be  arranged  in  such  a  way  as  to  avoid  "con- 
stellations" or  undesirable  collocations  of  any  sort;  and  the  dif- 
ficulties of  the  list  will  be  pretty  evenly  distributed. 

Two  other  points  were  considered  in  arranging  the  order  with- 
in the  lists.  When  the  test  is  given  with  a  time  limit,  it  is  es- 
pecially desirable  to  have  the  responses  of  uniform  difficulty  in 
that  part  of  the  list  where  most  of  the  subjects  will  be  stopped, 
so  that  there,  at  least,  the  single  words  shall  constitute  equal  units. 
We  have  therefore  placed  those  of  our  "pairs"  which  are  com- 
posed of  words  of  medium  difficulty  in  the  midst  of  the  list, 
from  about  the  8th  to  about  the  16th  word.  If  then  the  time 
limit  is  so  chosen  that  the  great  majority  of  subjects  shall  be 
stopped  in  this  part  of  the  list,  the  separate  words  may,  without 
much  error  on  the  average,  be  counted  as  equal  units. 

The  other  point  concerns  the  writing  of  responses.  In  reality, 
as  explained  in  the  introduction,  an  easy  association  test  is  very 
ill  adapted  for  written  responses,  because  the  time  of  writing 
is  much  greater  than  that  of  easy  association,  and  individual 
differences  in  speed  of  writing  altogether  mask  the  differences 
in  speed  of  association.  However,  in  case  of  the  opposites  test, 
we  have  determined  the  writing  times  for  the  correct  response 
and  so  distributed  the  stimuli  that  the  writing  times  for  the  two 
lists  of  twenty,  for  the  halves  of  each  list,  and,  as  nearly  as  possi- 
ble, for  the  pairs  throughout  each  half,  shall  be  equal.  This  has 
not  been  attempted  for  the  other  tests  given  below,  because  the 
response  words  are  not  wholly  determined  in  advance. 

i.  The  Opposites  test.  This  test  has  one  advantage  over  all 
the  others  in  the  series  of  logical-relation  tests,  namely  that  the 
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answers  can  be  definitely  scored  as  right  or  wrong.  Opposites 
apparently  are  the  most  available  material  for  a  test  of  completely 
controlled  association — with  the  exception,  indeed,  of  the  naming 
tests  and  of  the  arithmetical  tests  already  brought  forward.  For 
this  reason,  we  have  taken  unusual  pains  with  the  selection  and 
arrangement  of  material  for  this  test.  As  it  appears  possible  to 
select  forty  words  free  from  difficulty,  we  offer  two  lists  of 
twenty,  with  the  object  of  making  it  possible  to  give  two  equally 
difficult  tests  of  the  same  function.  Since,  however,  it  may  be 
desired  in  some  instances  to  have  the  very  simplest  material,  we 
also  present  a  list  of  twenty  "Easiest  Opposites",  all  of  which 
are  included  in  the  two  lists  of  twenty. 

The  lists  are  printed  on  three  separate  slips,1  in  12-point  type, 
well-leaded.  Lists  I.  and  II.  are  of  equal  difficulty,  and  the 
halves  of  each  list  are  equivalent,  as  far  as  can  be  judged  from 
the  results  of  their  use  so  far. 

The  instructions,  enforced  by  samples  (see  p — ),  require  the 
subject  to  respond  to  each  stimulus  word  by  the  word  having  the 
opposite  meaning;  as,  "long-short". 


/ 

long 
soft 
white 
far 
up 

smooth 
early- 
dead 
hot 
asleep 
lost 
wet 
high 
dirty 
east 
day 
yes 
wrong 
empty 
top 


OPPOSITES  TEST 
II 

north 

sour 

out 

weak 

good 

after 

above 

sick 

slow 

large 

rich 

dark 

front 

love 

tall 

open 

summer 

new 

come 

male 


Easiest 

high 

summer 

out 

white 

slow 

yes 

above 

north 

top 

wet 

good 

rich 

up 

front 

long 

hot 

east 

day 

big 

love 


aThe  type,  etc.,  of  these  tests  is  not  reproduced  here. 
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2.  The  verb-object  test.  More  good  stimulus  words  are  avail- 
able here  than  in  any  similar  test  except  that  for  opposites.  Here 
again  we  have  selected  two  equivalent  lists,  and  also  a  list  con- 
taining the  very  easiest  stimuli,  as  judged  from  results  with  about 
20  subjects.  The  verbs  are  to  be  treated  as  transitive,  and  ob- 
jects supplied;  for  example,  "sing  song",  "build  house.',  The 
audible  repetition  by  the  subject  of  the  stimulus  word  is  not  re- 
quired, and  may  interfere  somewhat  with  the  experimenter's  re- 
cord; but  it  does  not  change  the  times  to  an  appreciable  extent. 

VERB-OBJECT  TEST 


sing 

build 

wear 

shoot 

scold 

win 

answer 

weave 

wink 

mend 

pump 

learn 

open 

eat 

climb 

lend 

smoke 

singe 

dig 

sift 


II 
read 
tear 
throw 
paint 
mail 
light 
sail 
spin 
lock 
wash 
bake 
spill 
kiss 
polish 
sweep 
fill 

sharpen 
write 
chew 
drive 


Easiest 

wash 

sing 

bake 

read 

chew 

learn 

mail 

sweep 

scold 

wear 

sharpen 

kiss 

smoke 

answer 

climb 

lock 

throw 

sail 

dig 

wink 


3.  The  supraordinate  concept  or  species-genus  test.  The  in- 
structions are  to  name  a  class  to  which  the  given  object  belongs, 
or  to  "tell  what  sort  of  thing  each  is" ;  as  "oak — tree." 

4.  The  subordinate  concept  or  genus-species  test.  The  in- 
structions are  to  name  an  example  of  the  class  mentioned,  or 
to  "mention  a — ",  as,  "color — red". 

5.  The  part-whole  test.  The  instructions  are  to  name  the 
whole  thing  of  which  the  part  is  mentioned ;  as  "elbow — arm". 
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Supraordinate 

Subordinate 

Part-Whole 

Concept  Test 

Concept  Test 

Test 

oak 

color 

elbow 

measles 

holiday 

hinge 

July 

fish 

page 

shark 

tool 

finger 

quinine 

metal 

wing 

beef 

vegetable 

morning 

canoe 

coin 

blade 

banana 

city 

mattress 

Atlantic 

insect 

chimney 

Alps 

food 

cent 

penny- 

fruit 

sleeve 

dictionary 

disease 

brick 

cabbage 

grain 

deck 

Rhine 

drink 

France 

murder 

month 

pint 

dog 

ocean 

fin 

sparrow 

language 

steeple 

London 

river 

month 

football 

newspaper 

hub 

rose 

tree 

chin 
Attribute- 

Whole-Part 

Agent- Action 

Action-Agent 

Substance 

Test 

Test 

Test 

Test 

apple 

baby 

• 

gallops 

sharp 

clock 

fire 

bites 

hot 

knife 

dog 

boils 

dusty 

book 

laborer 

sleeps 

raw 

hat 

pencil 

floats 

deep 

pencil 

army 

growls 

ripe 

hand 

heart 

sails 

funny 

dog 

pin 

roars 

tall  " 

oyster 

gun 

scratches 

stormy 

church 

eyes 

stings 

new 

chair 

bird 

shoots 

hilly 

bird 

wind 

melts 

strong 

banana 

lungs 

swims 

muddy 

shoe 

bell 

explodes 

pretty 

train 

musician 

aches 

noisy 

finger 

parrot 

blows 

white 

house 

clock 

mews 

steep 

coat 

axe 

cuts 

round 

cart 

broom 

flies 

smoky 

face 

mosquito 

burns 

curly 
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6.  The  whole-part  test.  The  instructions  are  to  name  a  part 
of  each  thing  mentioned ;  as  "apple — core". 

7.  The  agent-action  or  subject-verb  test.  The  instructions  are 
to  put  an  appropriate  verb  to  each  noun  as  subject;  or  to  "tell 
what  each  of  these  does  or  can  do;"  as  "baby — cries". 

8.  The  action-agent  or  verb-subject  test.  The  instructions  are 
to  supply  a  subject  to  each  verb,  or  to  "tell  what  does  or  can  do 
each  of  these  things;"  as,  "horse  gallops." 

9.  The  attribute-substance  or  adjective-noun  test.  The  in- 
structions are  to  supply  an  appropriate  noun  for  each  adjective, 
or  to  "tell  something  that  is  or  may  be  each  of  the  following", 
or  to  complete  the  expression,  "A  good — ",  etc. ;  as,  "sharp  knife", 

10.  The  mixed  relations  test.  In  the  preceding  tests,  the  task 
remains  the  same  through  a  series  of  reactions;  in  the  present 
test  the  particular  relation  along  which  the  reaction  is  required 
to  occur  changes  with  each  reaction — the  object  being  to  get 
some  insight  into  flexibility  of  mental  performance.  We  were 
long  at  a  loss  for  some  means  of  indicating  the  new  task  without 
lengthy  explanations  at  each  new  stimulus  and  also  without 
the  use  of  such  technical  terms  as  supraordinate,  etc.  Finally  a 
device  used  by  one  of  us  previously  in  the  study  of  consciousness 
of  relations  seemed  to  meet  our  present  needs :  the  relation  along 
which  the  reaction  is  to  take  place  is  indicated  before  each  new 
stimulus  word  by  a  pair  of  words  serving  as  a  sample.  The  sub- 
ject is  to  note  the  relation  of  the  second  word  to  the  first,  and 
then  find  a  word  standing  in  this  same  relation  to  the  third  word. 
Thus,  in  the  example  "Box — square  Orange — ?"  "square"  gives 
a  quality  of  "box",  or,  more  specifically,  the  shape  of  the  box, 
and  it  is  required  to  mention  the  shape  of  an  orange;  in  the 
example  "East' — west  Day — ?"  since  east  and  west  are  opposites 
the  task  is  to  find  the  opposite  of  day ;  and  in  the  example,  "Penny 
— copper  Nail — ?"  the  task  is  to  mention  the  material  of  which 
the  nail  is  composed.  Some  of  the  relations  are  not  readily  nam- 
ed, but  little  difficulty  has  appeared,  with  the  adult  educated  sub- 
jects already  tested,  in  grasping  the  relation  from  the  sample 
given.       Instructions    for    this    test    must    proceed    largely    by 
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the  use  of  samples,  of  which  several  must  be  given,  in  order  that 
the  subject  may  realize  that  it  is  not  always  the  same  relation  that 
is  needed,  but  a  new  relation  each  time  as  indicated  by  the  first 
two  words  in  the  line. 


Mixed  Relatio 

ns   Test 

Mixed   Relatione 

?   Test 

I 

II 

Eye — see 

Ear — 

Good — bad 

Long- 

Monday — Tuesday 

April — 

Eagle — bird 

Shark — 

Do— did 

See — 

Eat — bread 

Drink — 

Bird — sings 

Dog- 

Fruit — orange 

Vegetable- 

Hour — minute 

Minute — 

Sit — chair 

Sleep — 

Straw — hat 

Leather — 

Double — two 

Triple — 

Cloud — rain 

Sun — 

England — London 

France — 

Hammer — tool 

Dictionary — 

Chew — teeth 

Smell — 

Uncle — aunt 

Brother — 

Pen — write 

Knife- 

Dog— puppy 

Cat- 

Water — wet 

Fire — 

Little — less 

Much — 

He — him 

She- 

Wash. — face 

Sweep — 

Boat — water 

Train — 

House — room 

Book- 

Crawl — snake 

Swim — 

Sky — blue 

Grass — 

Horse — colt 

Cow — 

Swim — water 

Fly- 

Nose — face 

Toe — 

Once — one 

Twice — 

Bad — worse 

Good — 

Cat — fur 

Bird- 

Hungry — food 

Thirsty — 

Pan — tin 

Table— 

Hat — head 

Glove — 

Buy — sell 

Come — 

Ship — captain 

Army — 

Oyster — shell 

Banana — 

Man — woman 

Boy— 

Results  with  the  logical  relations  test.  After  the  tests  had 
reached  practically  their  present  condition,  they  were  tried  with 
thirteen  college  and  graduate  students  (in  a  few  cases,  the  num- 
ber of  individuals  was  less  than  this).  In  these  experiments, 
lists  of  ten  stimulus  were  presented  visually,  but  the  time  of  the 
single  reactions  was  roughly  taken  by  the  device  mentioned  on 
page  17.  Usually  two,  and  in  the  case  of  the  opposites  and  verb- 
object  tests  four  lists  of  ten  were  used,  and  each  subject's  aver- 
age time  per  single  reaction  was  obtained.  The  averages  given 
in  the  accompanying  table  are  the  average  of  the  individual  aver- 
ages, and  the  A.D.  is  that  of  the  individual  averages  from  the 
general  average. 
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r.  per 

A. 

D.  of  indivs. 

Range 

e  reaction 

P.E. 

from  general  av. 

of  indivs. 

23 

.06 

.16 

1.03— 1.50 

.11 

.04 

.12 

0.85 — 1.40 

•39 

.05 

.19 

1.08— 1.75 

31 

.05 

.14 

1. 10— 1.55 

•54 

.07 

.31 

0.90 — 2.20 

.84 

.07 

•31 

1.20 — 2.63 

•53 

.06 

.27 

1.03—2.50 

•57 

.07 

•32 

1. 13— 2.35 

■30 

.03 

.12 

0.93—1.70 

•55 

.07 

•32 

1.03 — 2.68 

•53 

.07 

.28 

1.08—3.05 

.14 

13 

•53 

2.33—4.40 

Av 
singl< 

Opposites  1  and  2 1 

Opposites,  easiest   1 

Verb-obj.  1  and  2 1 

Verb-obj.,   easiest    1 

Suproord.  concept 1 

Subord.    concept 1 

Part-whole   1 

Whole-part   1 

Agent-action    1 

Action-agent   1 

Attrib.-subst 1 

Mixed  relations 3 

The  degree  of  agreement  between  the  results  of  the  several 
logical  relations  tests  is  a  matter  of  some  interest  as  indicating 
to  what  extent  a  single  test  is  a  fair  indication  of  the  individual's 
ability  in  this  whole  class  of  performances.  By  methods  which 
will  be  more  fully  described  in  another  paper,  we  have  determined 
the  average  standing  of  each  of  our  thirteen  subjects  in  the  nine 
logical  relations  tests  (excluding  the  mixed  relations  test),  and 
have  correlated  this  average  standing  with  the  standing  in  each 
single  test.  The  results  follow,  in  the  form  of  Pearson  coeffi- 
cients, uncorrected  for  attenuation. 


r  P.E. 

Correlation  of  Average  with:     Opposites -J-.88  .03 

Verb-object 4-.70  .08 

Subordinate   cone +.72  .07 

Supraordinate  concept....  +.91  03 

Part-whole    -f .86  .04 

Whole-part  +.76  .06 

Agent-action    +83  .04 

Action-agent    -K84  .04 

Attribute-substance   +-54  -12 

As  far  as  these  few  results  indicate,  then,  the  opposites  and 
supraordinate  concept  tests  seem  slightly  better  than  the  rest  as 
representative  of  this  general  sort  of  controlled  association. 
The  correlation  between  the  opposites  and  the  supraordinate 
concept  tests  was  +.70,  with  P.E.  of  .08,  while  the  average 
correlation  between  any  two  of  the  nine  logical  relations  tests 
is  +-57- 
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Comparative  speed  of  the  different  forms  of  controlled 

association 
It  may  be  of  interest,  since  results  are  available  in  several  tests 
from  comparable  and  highly  reliable  (though  not  numerous)  sub- 
jects, to  bring  together  the  times  per  single  reaction,  placing  them 
in  the  order  from  quickest  to  slowest.  In  all  these  cases,  a  series 
of  stimuli  was  simultaneously  presented,  so  that  overlapping  took 
place. 

Time  in  seconds 
Performance.  per  single  reaction.  P.E. 

Reading  letters  or  short  words 0.33 

Naming  colors  0.61  .03 

Naming  forms   0.89  .09 

Adding  two  one-place  numbers  (Kraepelin  blank  ) 1.07  .08 

Easiest  opposites i.u  .04 

Agent-action  1.30  .03 

Verb-object,  easiest  1.31  .05 

Adding  4  (const,  increment  test) 1.36  .08 

Part-whole 1.53  .06 

Attribute-substance    1.53  .06 

Supraordinate  concept 1.54  .07 

Action-agent    1.55  .07 

Whole-part 1.57  .07 

Subtracting  4  (const,  increment  test)    1.53  .07 

Subordinate  concept 1.84  .07 

Mixed  relations  3.14  .13 

Adding  17  (const,  increment  test)   3.90  .31 


These  differences  are  of  course  not  to  be  understood  as  mean- 
ing that  the  finding  of  opposites  is  always,  or  even  on  the  aver- 
age, quicker  than  the  finding  of  wholes  when  parts  are  given. 
The  times  for  opposites  that  are  by  no  means  recondite  or  unusual 
run  up  to  an  average  of  at  least  5  seconds  per  reaction.  It  would 
be  futile  to  attempt  to  determine  the  average  or  median  time  for 
all  opposites,  and  even  more  futile  to  make  such  an  attempt  in 
case  of  the  part-whole,  genus-species  and  many  other  relations; 
there  would  be  no  way  of  setting  an  upper  limit  to  the  difficulty 
of  the  single  stimulus  words.  Such  a  statement  as  that  the  mind 
passes  more  readily  from  species  to  genus  than  from  genus  to 
species  has  therefore  not  much  real  validity.    The  fact  simply  is, 
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as  far  as  our  results  are  concerned,  that  the  easiest  opposites  are 
easier  than  the  easiest  part-whole  associations,  etc. ;  and  by  "eas- 
iest" is  meant,  in  case  of  the  several  logical  relations  tests,  the 
twenty  easiest.  The  differences  between  the  speed  of  controlled 
associations  are  perhaps  mainly  dependent  on  the  factor  of  fre- 
quency in  past  experience,  and  especially  on  the  frequency  of 
linguistic  transitions.  Thus,  transitions  between  opposites  are  fre- 
quent in  common  speech,  and  many  pairs  of  opposites  thus  be- 
come verbally  associated  in  a  high  degree.  The  reproductive 
tendencies  in  case  of  the  most  commonplace  opposites  are  there- 
fore strong;  and  it  may  also  be  that  the  "mental  set",  or  "de- 
termining tendency",  is  better  drilled  in  case  of  finding  opposites 
than  in  many  other  sorts  of  logical  relation. 


VIII.     THE  UNDERSTANDING  OF  INSTRUCTIONS 

As  already  mentioned  (p.  20),  a  test  should  not  ordinarily  be 
begun  till  the  subject  certainly  understands  the  instructions;  other- 
wise the  time  measured  is  partly  occupied  with  grasping  the 
problem,  and  only  partly  with  its  execution.  Each  test  should,  as 
nearly  as  possible,  be  a  test  of  one  sort  of  performance.  But  it 
seemed  desirable  to  attempt  to  test  the  ability  to  understand 
instructions,  and  accordingly  efforts  were  made  to  prepare  a  test 
which  should  give  many  different  sets  of  very  simple  instructions, 
with  the  object  of  discovering  the  subject's  speed  in  apprehend- 
ing them.  After  much  experimenting,  the  following  were  pro- 
duced, and  the  test  was  named  the  directions  test..  This  test 
should,  we  believe,  be  given  as  a  list  or  continuous  test,  with 
rough  timing  also  of  the  single  reactions,  so  as  to  get  the  median 
as  well  as  the  average  time  of  response.  The  reactions  are  to 
be  made  with  a  pencil ;  and  the  test  can  very  well  be  made  with 
a  time  limit  as  well  as  with  an  amount  limit. 

The  conditions  which  it  was  sought  to  meet  in  the  test  ma- 
terial are  (1)  that  the  motor  response  should  be  very  simple 
and  quickly  performed;  (2)  that  the  instructions  should  be  very 
simple,  but  varied;  and  (3)  that  the  instructions  should  be  as 
concise  as  possible,  in  order  that  reading  time  might  not  be  the 
determining  factor. 

1.  Easy  directions  test.  Two  blanks  are  provided,  of  ap- 
proximately equal  difficulty,  according  to  the  results  so  far  in 
hand.    The  halves  are  also  approximately  equivalent. 

2.  Hard  directions  test.  The  object  here  is  to  complicate  the 
directions  somewhat,  by  calling  for  conditional  and  alternative 
responses,  etc.  The  blank  is  arranged  in  the  general  form  of  an 
Ebbinghaus  combination  test.  The  instructions  are  simply  to  fill 
in  the  blank  according  to  the  directions  in  it.  The  first  two  or 
three  directions  are  easy,  so  as  to  put  the  subject  on  the  right 
track.  The  remaining  units  within  the  blank  (except  the  last) 
are  so  chosen  as  not  to  be  very  unequal,  with  the  object  of  making 
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the  blank  available  for  use  with  a  time  limit.  It  can  not,  how- 
ever, be  claimed  for  this  test  that  it  is  as  well  worked  over  and 
standardized  as  the  others  in  this  series. 

Results  with  the  directions  tests.  Data  so  far  in  hand  are 
rather  meager,  eight  subjects  having  taken  the  easy  directions 
test  in  approximately  its  present  form,  and  six  subjects  the 
harder  test — all  educated  adults.    The  results  follow : 

DIRECTIONS  TESTS. 

Av.      P.E.  A.  D.  Range 

Easy  tests,  time  in  sees,  per  reaction..     3.60        .28  .92  2.30 — 5.70 

Hard  test,  time  for  whole  blank  107.6        6.0  18.4  76 — 134 


If  the  number  of  reactions  in  the  hard  test  is  counted  as  20 
(which  is  approximately  correct),  the  average  time  per  re- 
actions is  5.38  seconds;  the  reactions  are  no  doubt  slower 
in  this  than  in  any  other  of  the  tests  described  in  this  paper.  To 
judge  from  the  six  subjects  who  have  taken  both  the  easy  and 
the  hard  directions  test,  the  correlation  between  the  two  is  very 
high  (Pearson  1  =  +  .92). 


Cross  out  the  smallest  dot:         •     •     • 

Put  a  comma  between  these  two  letters:   G     H 

How  many  ears  has  a  cat  ? 

Make  a  line  across  this  line : 

Show  by  a  cross  which  costs  more :  a  hat  or  an  orange. 

Write  8  at  the  thinnest  part  of  this  line :     — — — — 

Write  any  word  of  three  letters. 

I  n  I 

Put  a  dot  in  one  of  the  white  squares  :         HHT^fflfl 
Cross  out  the  word  you  know  best:  fish,  brol,  matzig. 

Leave  this  just  as  it  is  :    "^ ^»  (q) 

Mark  the  line  that  looks  most  like  a  hill :        Vx       )  '        \ 

How  many  t's  are  there  in  twist  ?  # 

Dot  the  line  that  has  no  dot  over  it : 

Write  o  after  the  largest  number :  3    86     12 

Mark  the  name  of  a  large  city :  London,  painters 

Make  a  letter  Z  out  of  this :        / 

Join  these  two  lines:    

Write  s  in  the  middle  square  :        I I    I I    I | 

Write  any  number  smaller  than  10. 
Put  a  question  mark  after  this  sentence 


Cross  out  the  g  in  tiger. 

Write  2  between  the  two  dots:       #     0  , 

How  many  feet  make  a  yard  ? 

Write  +  over  the  longest  word  •   It  rained  yesterday. 

Put  a  dot  below  this  line :  

Write  the  sum  of  these  numbers :    f 

Make  a  boy's  name  by  adding  one  letter  to  Joh 

.«..„.**.«„  A  O  □ 

What  comes  next  after  D  in  the  alphabet  ? 

Write  7  in  the  largest  square:       LJ      | |      I | 

Cross  out  the  blackest  letter  in  TEXAS 

Write  g  on  the  egg-shaped  figure :  v^. )  K^y 

Make  two  dots  between  these  lines: 

Put  the  sign  =  where  it  belongs:   3  +  2      5. 

Write  here the  middle  letter  of  get. 

Put  a  nose  on  this  face :         *     ) 

XXX 

Add  a  cross  and  make  these  rows  equal :     X  X  X  X 

Put  a  dot  in  the  circle,  below  the  center  :      (    '   J 
Draw  a  line  around  the  three  dots:      •    ♦         •  •  • 
Cross  out  the  last  word  in  this  sentence. 


With  your  pencil  make  a  dot  over  any  one  of  these 
letters  F  G  H  I  J,  and  a  comma  after  the 
longest  of  these  three  words:  boy  mother  girl 
Then,  if  Christmas  comes  in   March,  make  a  cross  right 

here but  if  not,  pass  along  to  the  next  question,  and 

tell  where  the  sun  rises If  you  believe  that 

Edison  discovered  America,  cross  out  what  you  just 
wrote,  but  if  it  was  some  one  else,  put  in  a  number  to 

complete    this    sentence:       "A    horse    has feet." 

Write  yes,  no  matter  whether  China  is  in  Africa  or  not 

;  and  then  give  a  wrong  answer  to  this  question: 

"How  many  days  are  there  in  the  week?" 

Write  any  letter  except  £  just  after  this  comma,  and 
then  write  no  if  i  times  5  are  10 Now,  if  Tues- 
day comes  after  Monday,  make  two  crosses  here ; 

but  if  not,  make  a  circle  here or  else  a  square  here 

Be  sure  to  make  three  crosses  between  these 

two  names  of  boys:     George Henry.     Notice 

these   two    numbers:     3,     5.     If  iron    is    heavier    than 

water,  write  the  larger  number  here ,  but  if  iron 

is  lighter  write  the  smaller  number  here Show 

by  a  cross  when  the  nights  are  longer:     in  summer? 

in  winter? Give  the  correct  answer  to  this  ques- 
tion:    "Does    water  run    uphill?" and  repeat 

your  answer  here Do  nothing  here  (  5  -j-  7  = 

),   unless  you   skipped    the  preceding    question; 

but  write  the  first  letter  of  your  first  name  and  the  last 
letter  of  your  last  name  at  the  ends  of  this  line: 


IX.    THE  FREE  ASSOCIATION  EXPERIMENT 

Few  procedures  in  experimental  psychology  have  so  richly  re- 
warded their  investigators  with  the  possibilities  of  practical  ap- 
plication. In  ordinary  psychological  nomenclature,  it  is  the  "as- 
sociation" experiment  par  excellence.  Within  the  past  seven 
years  it  has  achieved,  and  bids  fair  to  hold  indefinitely  its  place 
in  the  foremost  rank  among  the  methods  of  individual  psycho- 
logy. The  body  of  work  that  has  gathered  about  it  is  probably 
greater  than  that  about  any  other  single  psychological  experi- 
ment, and  it  is  not  surprising  that  it  constitutes  one  of  the  best 
understood,  as  well  as  one  of  the  most  potentially  significant  of 
them. 

The  preliminary  task  of  standardization  is  to  provide  as  er- 
ror-free a  method  as  practicable,  but  the  main  object  of  stand- 
ardization is  to  afford  a  basis  for  making  comparisons  between 
different  individuals.  An  experimental  method  becomes  stand- 
ardized in  the  most  complete  sense  when,  given  a  proper  technique, 
it  is  possible  to  accurately  rate  individual  records  with  refer- 
ence to  an  empirical  scale.  None  of  the  "mental  tests"  possesses 
this  quality  to  a  degree  comparable  with  the  free  association  ex- 
periment, within  the  limits  of  the  English  language.  This  is  main- 
ly due  to  the  work  of  Kent  and  Rosanoff  which  established  a  defi- 
nite standard  of  normality  for  a  specific  association  material.1 
Within  the  bounds  of  its  application,  it  would  be  an  impertinence 
to  offer  as  "standard"  any  procedure  for  the  free  association 
test  other  than  the  one  which  these  authors  have  developed;  our 
first  endeavor  will  be  then  to  describe  this  experimental  material, 
and  to  indicate  what  seem  to  be  the  best  methods  for  its  ap- 
plication. 

The  Kent-Rosanoff  experiment  consists  of  one  hundred  or- 
dinary English  words  of  somewhat  varying  difficulty,  in  the 
order  given  on  the  opposite  page,  and  the  making  of  the  test 

xKent  and  Rosanoff,  A  Study  of  Association  in  Insanity,  Am.  Journal  of 
Insanity,  LXVII,  pp.  37-96  and  317-390. 
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requires,  according  to  the  subject,  from  ten  to  twenty-five 
minutes. 

This  experiment  was  made  by  Kent  and  Rosanoff  with  iooo 
normal  subjects,  and  the  responses  were  tabulated  to  each  in- 
dividual stimulus  word.  These  constitute  the  so-called  "Fre- 
quency Tables,"  and  their  use  is  to  determine  the  "value,"  in 
terms  of  its  frequency  coefficient,  for  any  reaction  or  series  of 
reactions  in  a  record  of  this  experiment.  After  each  response- 
word  in  the  tables  occurs  a  number,  which  is  the  number  of  times 
that  the  word  to  which  it  attaches  occurred  as  a  response  to  the 
stimulus  word  among  the  iooo  observations.  This  figure,  divided 
by  10,  is  taken  as  the  "value"  of  the  response.  Thus  the  "value" 
of  the  association  table-accommodation  is  .1,  because  it  was  given 
by  only  one  of  the  iooo  subjects;  that  of  table-chair  is  26.7,  being 
given  by  267  of  the  iooo  subjects,  that  of  dark-room  is  2.2,  that 
of  music-art  is  .7.  It  is  found,  then,  that  different  records  of  the 
test  show  marked  differences  in  the  "value"  or  usualness  of  the 
associative  responses.  By  means  of  these  frequency  tables,  the 
proper  "value"  is  assigned  to  all  reactions  obtained  from  the 
stimulus-words.  Any  one  wishing  to  work  with  this  experiment 
must  provide  himself  with  a  copy  of  the  tables/  which  it  is  im- 
possible to  reproduce  here.  Some  measure  of  central  tendency 
for  all  the  measures  should  be  taken,  and  the  distribution  of 
the  measures  indicates  the  median  to  be  preferable  for  this  pur- 
pose to  the  average,  aside  from  its  greater  ease  of  calculation. 

The  first  and  foremost  datum  of  the  Kent-Rosanoff  experi- 
ment is  an  empirical  measure  of  the  tendency  of  the  subject's 
train  of  thought  to  move  in  usual  or  individual  channels;  more 
accurately  speaking,  along  objective  or  subjective  lines.  A 
number  of  interpretational  questions  arise  in  connection  with 
this  finding,  which  seems  less  correlated  with  education  than 
with  temperament.  It  is  perhaps  the  best  objective  correlate 
of  temperament  at  present  to  hand,  but  the  matter  is  a  rather 
complicated  one,  more  suitable  for  separate  discussion.  Here 
need  be  emphasized  only  the  preciseness  and  objectivity  with 

1  American  Journal  of  Insanity,  LXVII,  pp.  48-90.     To  be  had  of  G.  E. 
Stechert  &  Co.,  New  York. 


1.  Table 

26.   Wish 

51.  Stem 

76.  Bitter 

2.  Dark 

27.  River 

52.  Lamp 

77.  Hammer 

3.  Music 

28.  White 

53.  Dream 

78.  Thirsty 

4.  Sickness 

29.  Beautiful 

54.  Yellow 

79.  City 

5.  Man 

30.  Window 

55.  Bread 

80.  Square 

6.  Deep 

31.  Rough 

56.  Justice 

81.  Butter 

7.  Soft 

32.  Citizen 

57.  Boy 

82.  Doctor 

8.  Eating 

33.  Foot 

58.  Light 

83.  Loud 

9.  Mountain 

34.  Spider 

59.  Health 

84.  Thief 

10.  House 

35.  Needle 

60.  Bible 

35.  Lion 

11.  Black 

36.  Red 

61.  Memory 

86.  Joy 

12.  Mutton 

37.  Sleep 

62.  Sheep 

87.  Bed 

13.  Comfort 

38.  Anger 

63.  Bath 

88.  Heavy 

14.  Hand 

39.  Carpet 

64.  Cottage 

89.  Tobacco 

15.  Short 

40.   Girl 

65.  Swift 

90.  Baby 

16.  Fruit 

41.  High 

66.  Blue 

91.  Moon 

17.  Butterfly 

42.  Working 

67.  Hungry 

92.  Scissors 

18.  Smooth 

43.  Sour 

68.  Priest 

93.  Quiet 

19.  Command 

44.  Earth 

69.  Ocean 

94.  Green 

20.  Chair 

45.  Trouble 

70.  Head 

95.  Salt 

21.  Sweet 

46.   Soldier 

71.  Stove 

96.  Street 

22.  Whistle 

47.  Cabbage 

72.  Long 

97.  King 

23.  Woman 

48.  Hard 

73.   Religion 

98.  Cheese 

24.  Cold 

49.  Eagle 

74.  Whiskey 

99.  Blossom 

25.  Slow 

50.  Stomach 

75.   Child 

100.  Afraid 
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which  it  is  possible  to  evaluate  an  experiment  of  such  intimate 
and  subjective  character. 

If  special  circumstances  render  it  desirable,  it  is  possible  to 
employ  only  a  larger  portion  of  the  hundred  words  for  deter- 
mination of  the  usualness  in  response,  substituting  for  the  re- 
mainder, words  adapted  to  the  special  situation  in  hand.  It 
would  be  desirable  indeed,  if  the  Kent-Rosanoff  experiment  were 
made  the  framework  of  all  experiments  for  Tatbestandsdi- 
agnostik,  the  individually  significant  words  being  either  added  to 
it  or  replacing  some,  not  over  10  or  15  per  cent,  of  its  con- 
stituents. To  deal  objectively  with  questions  of  Tatbestands- 
diagnostik  requires  a  number  of  precautions  in  the  construction 
of  the  special  series,  the  enumeration  of  which  would  be  out  of 
place  here,  and  which  are  fully  discussed  by  the  investigators 
of  this  application  of  the  method. 

Unfortunately,  determinations  of  the  "median  of  community" 
(i.  e.,  the  median  "value"  of  the  100  reactions  in  a  record)  have 
as  yet  been  made  in  only  a  limited  nnmber  of  subjects.  In  some 
pathological  cases  it  would  become  indefinitely  small;  the  lowest 
median  ever  observed  by  either  of  the  writers  in  a  normal  sub- 
ject is  .7.  Such  a  figure  would  mean  that  half  the  reactions  of 
this  record  were  of  a  frequency  below  that  of  the  reaction  music- 
art  quoted  above.  The  other  extreme  of  the  range,  so  far  as 
observed,  is  18.2,  i.  e.,  half  of  the  reactions  in  such  a  record  are 
more  common  than,  i.  e.,  music-piano.  The  general  average 
value  of  the  reactions  in  the  above  mentioned  records  lies  not 
far  from  9.0,  that  is,  about  the  frequency  of  a  reaction  such  as 
mountain-valley. 

The  present  experimental  method  is  placed  under  one  disad- 
vantage to  a  much  greater  degree  than  other  association  tests; 
its  material  cannot  be  repeated  within  an  ordinarily  practicable 
time  save  under  greatly  changed  essential  conditions.  One  can 
foresee  that  circumstances  may  arise  in  which  a  comparative 
study  with  material  of  greater  extent  is  desirable.  Provision  is 
here  made  for  such  material  to  be  available,1  but  with  a  change  in 
the  character  of  the  material  comes  inevitably  a  change  in  the 
method  of  evaluation.    Beyond  the  range  of  the  frequency  tables 

1  See  Appendix,  pp.  80  ff. 
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one  must  fall  back  on  the  quasi-logical  system  of  classifying  the 
associations  that  was  practically  the  sole  means  of  dealing  with 
such  material  until  the  data  on  statistical  frequency  were  com- 
piled. The  proper  function  of  the  test,  however,  is  the  same  as 
before,  and  so  is  the  object  of  its  evaluation:  the  measure  of 
egocentricity  in  the  responses. 

There  is  no  need  to  fully  repeat  the  remarks  in  a  previous  con- 
tribution regarding  the  method  of  evaluation  that  seems  best 
adapted  to  these  conditions.  It  is  a  five-fold  classification,  in- 
cluding categories  termed  (1)  the  egocentric,  (2)  the  supraor- 
dinate,  (3)  the  contrast,  (4)  the  miscellaneous  or  "internal  ob- 
jective," and  (5)  the  speech-habit.1 

For  ordinary  purposes  of  comparison,  the  principal  question 
concerns  the  number  of  reactions  that  fall  into  the  category  of 
the  egocentric ;  and  a  large  or  small  number  of  such  associations 
is  subject  to  analogous  interpretations  with  the  empirically  de- 
termined tendency  towards  common  or  individualized  responses. 
xThe  definitions  and  illustrations  of  the  categories  may  be  summarized 
from  the  previous  paper  as  follows : 

1.  The  egocentric  reactions  may  be  typified  by — 

a.  Predicate  reactions.  Cloud-ominous,  flower-pretty,  crooked-line,  red- 
rose,  scratch-cat,  lion-roar,  money-wish,  invent-machine,  weasel-stealth,  beau- 
ty-rose, safe-quite,  almost-grown,  sing-well,  never-decide,  nicely-very  (includ- 
ing the  responses  yes  and  no). 

b.  Responses  in  the  form  of  proper  names.  Citizen-New  York,  boy- 
Johnny,  mountain-Kearsarge. 

c.  Reactions  interpreting  the  stimulus  word  as  a  proper  name.  Eagle- 
newspaper,  park-square. 

d.  Reaction  involving  the  response  of  a  pronoun.    Hand-you,  health-me. 

e.  Interjections,  failures  of  response  or  repetitions  of  the  stimulus  word. 

2.  The  supraordinate  category  is  confined  strictly  to  the  individual-genus 
order,  defined  in  such  examples  as,  priest-man,  potato-vegetable,  lily-flower, 
cow-animal. 

3.  The  contrast  group  is  composed,  of  course,  of  reactions  in  which  the 
response  meets  the  opposite  of  the  stimulus  and  is  made  up  of  such  associa- 
tions as,  good-bad,  trouble-pleasure,  scatter -gather ■_,  fertile-sterile,  and  the 
like. 

4.  The  miscellaneous  category  is  composed  essentially  of  the  remaining 
reactions  of  the  "inner"  type.  It  includes  about  45%  of  all  associations. 

5.  The  speech-habit  group  is  composed  of  associations  by  familiar  phrase 
(stand-pat),  word  compounding  (play-ground),  simple  sound  associations 
(tease-sneeze)  and  syntactic  changes  (high-height).  (Psychol.  Review,  1911, 
18,  229-288.) 
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The  egocentric  is  the  most  variable  of  the  five  categories,  ranging 
from  next  to  none  to  more  than  half  of  the  total  reactions  in  a 
single  experimental  series. 

With  respect  to  timing  the  associations,  the  stopwatch  is  the 
almost  universal  method,  and  must  be  accepted  as  such,  until 
some  more  accurate  procedure  is  devised.  Significant  differences 
are  usually  coarse  enough  to  be  manifested  in  measures  of  no 
greater  precision  than  this.  More  uniformity  in  the  manipulation 
is  desirable;  at  present,  some  operators  start  the  watch  on  the 
accented  syllable,  others  at  the  first  syllable,  of  the  simulus  word. 
The  watch  should  always  be  stopped  at  the  first  indication  of 
response,  even  if  it  does  cause  occasional  failure  of  timing  through 
the  subject's  clearing  his  throat.1 

Individual  differences  in  association  time  should  be  discussed 
from  the  standpoint  of  the  distribution  rather  than  any  single 
measure.  The  median  is  rather  preferred  as  a  measure  of  central 
tendency,  though  for  practical  purposes,  its  advantages  over  the 
average  are  of  less  account  here  than  in  most  cases  of  skew 
distribution.  The  presence  of  many  and  exclusively  long  meas- 
ures happens  to  be  more  important  here  than  in  most  similar 
series  of  measures.  Jung  has  proposed  a  special  comparison  of 
the  average  and  median;  this  is  a  convenient  statement  of  the 
distribution,  but  it  is  not  an  index  of  emotivity,  beyond  the 
limited  extent  to  which  the  association  time  can  be  interpreted 
in  this  direction. 

The  usual  instruction  in  the  free  association  test  is  that  the 
subject  shall  reply  with  the  first  word  the  stimulus  suggests  to 
him,  but  in  ordinary  practise  this  is  not  rigidly  enforced,  it  being 
sometimes  possible  to  derive  elements  of  special  significance  from 
factors  that  determine  the  subject's  departure  from  the  set  in- 
structions. For  comparison  with  the  frequency  tables,  it  is 
readily  apparent  that  the  single  word  response  must  be  rigidly 
required  in  the  Kent-Rosanoff  experiment;  in  cases  of  derelic- 
tion from  this  rule,  it  is  the  practise  of  these  authors  to  repeat 

1  Coughing  at  such  times  has  received  some  notice  as  a  Komplexmerkmal, 
though  it  has  been  sagely  remarked  that  this  loses  much  of  its  significance 
if  the  subject  in  question  has  a  cold. 


FREE  ASSOCIATION  EXPERIMENT  79 

the  stimulus  word  at  the  end  of  the  experiment,  in  order  to 
obtain  a  reaction  of  the  required  character. 

Presentation  and  response  have  nearly  always  been  oral  in 
this  experiment,  and  there  is  good  reason  to  make  no  change  in 
them.  Experimenters  differ  as  to  the  manner  in  which  they 
modulate  the  voice  to  the  test.  Some  experimenters,  as  Dr. 
Rosanoff,  speak  the  stimulus  word  with  a  rising  inflection,  as 
though  asking  a  direct  question,  some  as  an  exclamation,  as 
though  endeavoring  to  hurl  the  subject's  "complexes"  in  his 
teeth,  others  in  a  monotone.  There  are  subjects  whose  responses 
might  be  somewhat  affected  by  these  differences  in  procedure. 
This  is  one  of  the  minor  features  of  the  test,  in  which,  though 
uniformity  among  different  investigators  is  scarcely  practicable, 
the  experimenter  should  at  least  strive  to  keep  his  own  procedure 
constant. 

And  while  it  is  not  proven,  it  is  most  probable  that  the  re- 
sponses are  also  affected  by  the  personality  of  the  operator.  In 
making  fine  comparisons  between  records  by  different  experi- 
menters this  fact  must  be  borne  continually  in  mind. 

So  far  as  developed,  the  continuous  form  of  the  free  associa- 
tion experiment  is  a  method  of  possibilities  rather  than  of 
promise.  It  would  probably  be  capable  of  performing  many  of 
the  functions  of  the  discrete  form,  but  there  are  external  rea- 
sons why  it  would  not  be  likely  to  perform  them  so  well.  The 
problem  of  standardization  would  be  practically  confined  to  the 
development  of  the  most  significant  scheme  of  evaluation  after 
the  pattern  indicated  by  previous  workers  with  the  test.  It 
might  be  possible  to  employ  the  scheme  of  evaluation  proposed 
for  the  discrete  free  association  experiment,  each  single  word 
given  being  allowed  to  serve  as  the  stimulus  word  for  the  next 
association.  So  far  as  normal  psychology  is  concerned  the 
method  has  thus  far  dealt  very  largely  with  group  averages. 
As  a  method  of  individual  psychology  it  may  assume  a  position 
comparable  to  its  better  known  congener  only  as  a  result  of 
extended  and  laborious  researches. 

Note:  Blanks  for  all  the  preceding  tests  are  to  be  obtained  from  the 
C.   H.   Stoelting  Company,   1 13-125   North   Green   St.,   Chicago. 


APPENDIX 

The  following  series  of  iooo  words  is  intended  for  general  use 
in  the  free  association  experiment.  It  is  a  revision  of  the  series 
employed  in  the  experiments  on  the  practise  effects  in  the  test, 
and  is  modified  in  the  manner  suggested  by  the  experience  of  this 
investigation.  It  is  intended  to  contain  iooo  different  words, 
none  over  three  syllables,  so  far  as  possible  familiar  and  un- 
ambiguous. It  is  not  far  from  exhausting  the  total  available 
number  of  such  stimulus  words.  Ambiguous  stimulus  words 
have  a  special  and  useful  purpose,  but  not  in  a  test  of  the  present 
character.  The  details  of  the  preparation  of  this  list  were  sub-, 
stantially  the  same  as  in  the  previous  list,  save  in  one  particular. 
The  division  into  twenty  series  of  fifty  words  each  is  followed 
here.  But  the  present  list  also  contains  the  hundred  words  of 
the  Kent-Rosanoff  series,  distributed  pro  rata,  five  words  in  each 
series,  and  in  their  actual  order  of  sequence  in  the  Kent-Rosanoff 
test;  otherwise  their  arrangement  in  the  series  is  random,  save 
that  none  occur  in  the  first  ten  words  of  a  series.  In  the  list  as 
printed,  the  words  from  the  Kent-Rosanoff  series  are  distin- 
guished from  the  remainder  by  an  asterisk,  and  the  associations 
of  these  words  may  be  evaluated  by  the  frequency  tables.  In 
using  single  series  of  fifty  words  it  is  recommended  that  a  sheet 
of  paper  of  fifty  lines  be  obtained  upon  which  several  records 
of  reactions  to  the  same  stimuli  may  be  conveniently  noted. 
Stimulus  words  not  evoking  a  reaction  according  to  instructions 
may  be  repeated  at  the  close  of  the  series ;  and  if  a  stimulus  word 
evokes,  as  a  response,  the  word  coming  next  in  the  series,  this 
word  is  omitted  and  given  at  the  close. 

The  complete  list  is  as  follows: 


WORD  LIST 

I 

II 

III 

IV 

bottle 

drink 

locust 

weary 

produce 

captain 

divide 

tooth 

rope 

cedar 

restore 

practise 

delicate 

mischief 

tempt 

supper 

thick 

clean 

fade 

fun 

end 

also 

cheap 

pepper 

omelet 

path 

compel 

best 

expensive 

ride 

power 

heart 

cap 

salute 

baker 

island 

barrel 

grocery 

athlete 

machine 

burglar 

bashful 

♦black 

pit 

design 

true 

roof 

♦fruit 

cry 

perverse 

cradle 

return 

hip 

occasion 

certain 

marriage 

overcoat 

nuisance 

travel 

marsh 

freeze 

♦deep 

impress 

owl 

♦table 

pinch 

daughter 

water 

lightning 

satisfy 

gun 

summer 

follow 

tank 

book 

copper 

parlor 

hat 

barber 

beetle 

smoke 

nourish 

natural 

statue 

stretch 

sister 

elephant 

clothes 

tar 

♦soft 

ostrich 

oblong 

snake 

ham 

curse 

♦butterfly 

purpose 

ugly 

♦mutton 

constable 

♦dark 

age 

haste 

cloud 

unfair 

glory 

lizard 

collapse 

ditch 

tough 

result 

solid 

tiger 

acid 

nonsense 

number 

♦music 

♦eating 

index 

goose 

wicked 

crowd 

fool 

railroad 

prefer 

discourse 

dense 

excite 

fish 

watchful 

life 

hornet 

instrument 

indecent 

wine 

♦smooth 

guilty 

exchange 

♦comfort 

delay 

seed 

costume 

fever 

begin 

♦sickness 

style 

infirm 

cat 

crush 

trap 

comb 

asylum 

rich 

♦mountain 

spioe 

knee 

hash 

drift 

starch 

tight 

unseen 

crime 

venture 

car 

death 

cover 

♦hand 

^command 

umbrella 

abuse  (v) 

pirate 

insect 

blood 

open 

brandy 

hope 

gift 

♦house 

dress 

insist 

♦man 

enjoy 

pebble 

*chair 

allow 

untrue 

adventure 

star 

tailor 

dismay 

lip 

ice 

prospect 

unburden 

♦short 

picture 

school 

again 

pint 

bind 
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V 

VI 

VII 

VIII 

room 

crown 

forget 

pot 

pencil 

get 

goat 

camp 

dig 

honest 

pulse 

shirt 

indiscreet 

vacant 

unwholesome 

chain 

restless 

beechnut 

attention 

adult 

simple 

splinter 

dairy 

violin 

measure 

unbelief 

boast 

reason 

loss 

argue 

color 

excuse  (n) 

reckless 

conflict 

chin 

roast 

flirt 

alike 

servant 

ng 

pause 

skin 

dislike 

face 

prosper 

inside 

dead 

♦red 

*sweet 

♦wish 

♦rough 

common 

avoid 

hero 

fortune 

complexion 

fresh 

scarlet 

candy 

deserve 

real 

lamb 

perfect 

dim 

potato 

neck 

disdain 

view 

♦whistle 

clasp 

fierce 

♦sleep 

bite 

spear 

constant 

dirt 

clear 

♦river 

violent 

shoe 

eternal 

ox 

care 

slave 

jealous 

serious 

indeed 

protect 

barn 

garter 

♦citizen 

sting 

♦woman 

key- 

death 

funny 

persuade 

conquest 

sparrow 

solemn 

merit 

*white 

♦foot 

little 

receive 

scratch 

over 

♦anger 

above 

cool 

bother 

ramble 

conceal 

correct 

forward 

family 

revolt 

paste 

prepare 

annoy 

♦cold 

uncertain 

establish 

confusion 

join 

pudding 

gold 

ripe 

tender 

♦beautiful 

along 

greasy 

offense 

bacon 

cannon 

admire 

guide 

rancid 

boat 

cup 

prompt 

fertile 

song 

insult 

floor 

dog 

♦spider 

easy 

advance 

perfume 

another 

impudent 

bundle 

toy- 

irony 

♦carpet 

ignorant 

dust 

art 

decay 

*slow 

pansy- 

dove 

*girl 

blunder 

lake 

poem 

announce 

confidence 

♦window 

herald 

prudent 

knob 

vow 

♦needle 

trumpet 

flesh 

pancake 

treasure 

convenient 

future 

cork 

sensitive 

ask 

wart 

gay 

fog 

supreme 

market 

feast 

tunnel 

portable 

chocolate 

gem 

remain 

before 

disease 

enormous 

closet 

alone 

WORD 

LIST 

IX 

X 

XI 

XII 

accuse 

crab 

uphold 

intimate 

flame 

cart 

pickle 

able 

clown 

riot 

food 

suspect 

edge 

preserve 

raven 

barley 

frost 

preach 

unwell 

attack 

rust 

unclean 

ready- 

dishonor 

corrupt 

steep 

away 

accident 

appear 

master 

blame 

betray 

caution 

sulphur 

competent 

door 

poverty- 

laugh 

mark 

prince 

curtain 

imp 

improve 

♦justice 

army 

invite 

♦stem 

aim 

♦high 

raisin 

raw 

revenge 

people 

minnow 

defy 

active 

polish 

promise 

brook 

purple 

♦working 

ungracious 

vile 

decoy 

almost 

good 

storm 

noise 

idea 

♦soldier 

refined 

fable 

cask 

mask 

thankful 

unsafe 

alcohol 

money- 

fast 

fame 

minute 

doll 

tremble 

strength 

stain 

rotten 

center 

scoff 

nurse 

♦cabbage 

♦lamp 

humble 

♦sour 

cost 

saddle 

♦boy 

chapel 

irksome 

pin 

interest 

plant 

apricot 

denounce 

old 

sky- 

stone 

cook 

wealthy 

forest 

♦hard 

fraud 

modest 

companion 

brute 

bring 

♦light 

repeat 

escape 

paint 

fact 

against 

gain 

nut 

violet 

finish 

trifle 

immense 

appetite 

sermon 

♦eagle 

♦dream 

attraction 

♦earth 

admit 

condition 

♦health 

conceit 

snow 

descend 

across 

crack 

about 

splash 

piano 

drag 

equal 

abroad 

least 

condemn 

brown 

♦yellow 

salmon 

iron 

late 

deceive 

price 

emperor 

adore 

bride 

garden 

plead 

perish 

worship 

scar 

apart 

tomb 

infamous 

burn 

ivy 

harsh 

drop 

ashamed 

♦trouble 

♦stomach 

♦bread 

♦bible 

mouse 

wasp 

backwards 

deny 

event 

unripe 

pattern 

quantity 

claw 

friend 

cliff 

idle 

ingenious 

taste 

level 

wash 

minister 

joke 

body 

reproach 

impose 

propose 

elevate 

energy 
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XIV 

XV 

XVI 

alarm 

astonish 

contented 

find 

distrust 

vest 

praise 

broom 

dimple 

whale 

pump 

dagger 

bounce 

outfit 

poem 

love 

twig 

recover 

tennis 

try 

indulge 

embrace 

guard 

lazy 

run 

devil 

cake 

arrow 

agree 

game 

calm 

come 

secret 

towel 

remorse 

success 

ache 

faithful 

play 

coy 

advantage 

dwarf 

mouth 

pure 

napkin 

use  (v) 

linen 

shower 

hill 

north 

♦stove 

jump 

shelter 

thread 

belt 

♦bitter 

name 

rejoice 

amuse 

uproar 

injure 

*blue 

sign 

catch 

*memory 

disaster 

bag 

contest 

finger 

keep 

concert 

empty 

emblem 

handsome 

♦long 

unhappy 

spool 

rescue 

absent 

divine 

unfit 

audacious 

maiden 

♦hammer 

middle 

cage 

twist 

feather 

*sheep 

honey 

false 

disorder 

outrage 

guess 

plunge 

naughty 

accept 

disgrace 

murder 

exacting 

low 

shark 

♦religion 

abandon 

ardent 

flannel 

magic 

*thirsty 

*bath 

busy 

believe 

pay 

emerald 

unmarried 

author 

increase 

wagon 

angel 

oil 

♦city 

stun 

hospital 

choke 

chase 

gentle 

secure 

♦whiskey 

unemployed 

dodge 

♦hungry 

silver 

rhyme 

*cottage 

jewel 

noble 

map 

shock 

nice 

breast 

wretched 

contrary 

carve 

person 

distance 

hunt 

provoke 

influence 

playful 

sin 

alive 

magnet 

impulse 

asleep 

♦priest 

glad 

land 

exquisite 

orange 

ink 

moderate 

sweat 

battle 

introduce 

velvet 

change 

tube 

profane 

mix 

*swift 

♦ocean 

winter 

parent 

expert 

apology 

help 

♦square 

quality 

inch 

repress 

separate 

instant 

pretty 

*child 

sonnet 

progress 

brick 

field 

trade 

melt 

verse 

egg 

nest 

applause 

♦head 

rat 

fancy 

cream 

bad 

mock 

bench 
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XVII 

XVIII 

XIX 

XX 

grief 

decorate 

contrast 

demon 

parsnip 

chance 

unhurt 

credit 

purse 

sack 

fix 

frolic 

unlikely 

scold 

interval 

include 

walk 

portly 

fond 

rascal 

hod 

sorrow 

grain 

pardon 

comrade 

month 

mistake 

soap 

thought 

painful 

front 

fear 

lemon 

quarrel 

quart 

arise 

refuse 

flower 

lecture 

cane 

paper 

suffer 

♦moon 

destroy 

cause 

fault 

usher 

chart 

pocket 

*Joy 

brier 

refresh 

task 

cab 

fountain 

♦street 

kit 

discord 

church 

wrong 

♦butter 

sponge 

attempt 

rattle 

pie 

mother 

feed 

♦king 

strong 

♦bed 

tame 

send 

♦doctor 

den 

medicine 

glow 

Pig 

support 

glove 

raft 

punish 

conscience 

dispute 

mercy 

regiment 

devotion 

♦scissors 

dinner 

walnut 

difficult 

evil 

♦cheese 

weather 

adorn 

irritate 

bless 

♦loud 

immoral 

advice 

drive 

remove 

spite 

neighbor 

scorn 

fling 

brave 

cravat 

enter 

compare 

circle 

♦quiet 

scorch 

queen 

lettuce 

entire 

expression 

same 

ivory 

contempt 

♦blossom 

war 

urge 

talk 

ghost 

play 

imagine 

touch 

aboard 

exercise 

infinite 

flag 

parcel 

grind 

observe 

anxious 

dreadful 

♦thief 

assist 

hurt 

small 

pretend 

beast 

♦green 

oppose 

knock 

wheat 

crumb 

queer 

orchard 

♦heavy 

escort 

reduce 

president 

repose 

notch 

reward 

decent 

terror 

bird 

outlaw 

croak 

under 

husband 

♦afraid 

plaster 

caress 

control 

drum 

lump 

lard 

stable 

clover 

question 

learn 

♦salt 

intellect 

lend 

♦tobacco 

ornament 

elbow 

around 

destiny 

errand 

milk 

merry 

fire 

hoop 

smell 

♦lion 

consent 

blush 

scandal 

awake 

♦baby 

dull 

happy 

sacred 

excellent 

many 

mill 

